Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving

Kemiao Huang; Qi Hao

IROS 2021poster91 citations

Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving

Abstract

Multi-object tracking (MOT) with camera-LiDAR fusion demands accurate results of object detection, affinity computation and data association in real time. This paper presents an efficient multi-modal MOT framework with online joint detection and tracking schemes and robust data association for autonomous driving applications. The novelty of this work includes: (1) development of an end-to-end deep neural network for joint object detection and correlation using 2D and 3D measurements; (2) development of a robust affinity computation module to compute occlusion-aware appearance and motion affinities in 3D space; (3) development of a comprehensive data association module for joint optimization among detection confidences, affinities and start-end probabilities. The experiment results on the KITTI tracking benchmark demonstrate the superior performance of the proposed method in terms of both tracking accuracy and processing speed.

BibTeX

@inproceedings{iros2021_jointmultiobject,
  title = {Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving},
  author = {Kemiao Huang and Qi Hao},
  booktitle = {IROS 2021},
  year = {2021}
}