SDTrack:Spatially decoupled tracker for visual tracking

Abstract

Recent models based on encoder-decoder architecture have shown excellent performance in visual object tracking. The encoder models the global spatiotemporal feature correlation between the template and the search regions, while the decoder learns query embeddings to predict the spatial location of the target. However, in previous methods, decoders are query-shared, which may lead to suboptimal results. We observe that different regions in the visual feature map are suitable for performing different tasks. Salient regions in object provide important information for classification task, while the boundaries around it are more beneficial for box localization task. We therefore propose a spatially decoupled tracker called SDTrack. The tracker contains a query selection module that we carefully design to select appropriate queries for both classification and regression tasks. We divide the cross-attention module in the decoder and add the box-to-pixel relative position offset (BoxRPB) term to the cross-attention, so that the attention is more focused on the respective areas of interest while introducing smaller overhead. Finally, we propose an alignment loss to solve the misalignment problem between accurate classification and precise localization, further improving tracking performance. Through extensive experiments, we demonstrate that SDTrack achieves new SOTA performance on multiple benchmarks compared to previous work, while running at real-time speeds.