STORM: Segment, Track, and Object Re-Localization from a Single Image
Yu Deng, Teng Cao, Hikaru Shindo, Quentin Delfosse, Jiahong Xue, Kristian Kersting
Abstract
Accurate 6D pose estimation and tracking are core capabilities for physical AI systems, yet real-world deployment remains brittle and labor-intensive. Many pipelines rely on CAD models, manual masking, or per-object adaptation, and still fail under occlusion or fast motion without a principled way to recognize failure. We propose STORM, a unified framework for reference-conditioned 6D tracking with minimal manual input and improved robustness. STORM introduces two mechanisms: (i) Hierarchical Spatial Fusion Attention (HSFA), which performs latent manifold alignment between reference and query features, guided by vision-language semantic conditioning to resolve instance ambiguities; and (ii) an energy-based failure detector to detect drift and trigger automatic re-initialization, yielding a self-healing tracker. Experiments on LM-O and YCB-Video show that STORM improves annotation-free pose tracking accuracy over strong baselines and recovers reliably from severe occlusions and rapid viewpoint changes with minimal overhead.
BibTeX
@inproceedings{
deng2026storm,
title={{STORM}: Segment, Track, and Object Re-Localization from a Single Image},
author={Yu Deng and Teng Cao and Hikaru Shindo and Quentin Delfosse and Jiahong Xue and Kristian Kersting},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=oqUjAjGQY3}
}