Accurate Target Annotation in 3D from Multimodal Streams

Oswald Lanz; Alessio Brutti; Alessio Xompero; Xinyuan Qian; Maurizio Omologo; Andrea Cavallaro

ICASSP 2019accepted0 citations

Accurate Target Annotation in 3D from Multimodal Streams

Oswald Lanz, Alessio Brutti, Alessio Xompero, Xinyuan Qian, Maurizio Omologo, Andrea Cavallaro

Abstract

Accurate annotation is fundamental to quantify the performance of multi-sensor and multi-modal object detectors and trackers. However, invasive or expensive instrumentation is needed to automatically generate these annotations. To mitigate this problem, we present a multi-modal approach that leverages annotations from reference streams (e.g. individual camera views) and measurements from unannotated additional streams (e.g. audio) to infer 3D trajectories through an optimization. The core of our approach is a multi-modal extension of Bundle Adjustment with a cross-modal correspondence detection that selectively uses measurements in the optimization. We apply the proposed approach to fully annotate a new multi-modal and multi-view dataset for multi-speaker 3D tracking.

BibTeX

@inproceedings{icassp2019_accuratetargetan,
  title = {Accurate Target Annotation in 3D from Multimodal Streams},
  author = {Oswald Lanz and Alessio Brutti and Alessio Xompero and Xinyuan Qian and Maurizio Omologo and Andrea Cavallaro},
  booktitle = {ICASSP 2019},
  year = {2019}
}