S.T.A.R.-Track: Latent Motion Models for End-to-End 3D Object Tracking With Adaptive Spatio-Temporal Appearance Representations
Simon Doll, Niklas Hanselmann, Lukas Schneider, Richard Schulz, Markus Enzweiler, Hendrik P. A. Lensch
Abstract
Following the tracking-by-attention paradigm, this letter introduces an object-centric, transformer-based framework for tracking in 3D. Traditional model-based tracking approaches incorporate the geometric effect of object- and ego motion between frames with a geometric motion model. Inspired by this, we propose STAR-TRACK, which uses a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">latent motion model</i> (LMM) to additionally adjust object queries to account for changes in viewing direction and lighting conditions directly in the latent space, while still modeling the geometric motion explicitly. Combined with a novel learnable track embedding that aids in modeling the existence probability of tracks, this results in a generic tracking framework that can be integrated with any query-based detector. Extensive experiments on the nuScenes benchmark demonstrate the benefits of our approach, showing <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">state-of-the-art</i> (SOTA) performance for DETR3D-based trackers while drastically reducing the number of identity switches of tracks at the same time.
BibTeX
@inproceedings{ral2024_startracklatentm,
title = {S.T.A.R.-Track: Latent Motion Models for End-to-End 3D Object Tracking With Adaptive Spatio-Temporal Appearance Representations},
author = {Simon Doll and Niklas Hanselmann and Lukas Schneider and Richard Schulz and Markus Enzweiler and Hendrik P. A. Lensch},
booktitle = {RA-L 2024},
year = {2024}
}