Longshortnet: Exploring Temporal and Semantic Features Fusion In Streaming Perception
Chenyang Li, Zhi-Qi Cheng, Jun-Yan He, Pengyu Li, Bin Luo, Han-Yuan Chen, Yifeng Geng, Jin-Peng Lan
Abstract
Streaming perception is a fundamental task in autonomous driving that requires a careful balance between the latency and accuracy of the autopilot system. However, current methods for streaming perception are limited as they rely only on the current and adjacent two frames to learn movement patterns, which restricts their ability to model complex scenes, often leading to poor detection results. To address this limitation, we propose LongShortNet, a novel dual-path network that captures long-term temporal motion and integrates it with short-term spatial semantics for real-time perception. Our proposed LongShortNet is notable as it is the first work to extend long-term temporal modeling to streaming perception, enabling spatiotemporal feature fusion. We evaluate LongShortNet on the challenging Argoverse-HD dataset and demonstrate that it outperforms existing state-of-the-art methods with almost no additional computational cost. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
BibTeX
@inproceedings{icassp2023_longshortnetexpl,
title = {Longshortnet: Exploring Temporal and Semantic Features Fusion In Streaming Perception},
author = {Chenyang Li and Zhi-Qi Cheng and Jun-Yan He and Pengyu Li and Bin Luo and Han-Yuan Chen and Yifeng Geng and Jin-Peng Lan and Xuansong Xie},
booktitle = {ICASSP 2023},
year = {2023}
}