JST: Joint Self-training for Unsupervised Domain Adaptation on 2D&3D Object Detection

Abstract

2D&3D object detection always suffers from a dramatic performance drop when transferring the model trained in the source domain to the target domain due to various domain shifts. In this paper, we propose a Joint Self-Training (JST) framework to improve 2D image and 3D point cloud detectors with aligned outputs simultaneously during the transferring. The proposed framework contains three novelties to overcome object biases and unstable self-training processes: 1) an anchor scaling scheme is developed to efficiently eliminate the object size biases without any modification on point clouds; 2) a 2D&3D bounding box alignment method is proposed to generate high-quality pseudo labels for the self-training process; 3) a model smoothing based training strategy is developed to reduce the training oscillation properly. Experiment results show that the proposed approach improves the performance of 2D and 3D detectors in the target domain simultaneously; especially the superior accuracy of 3D detection can be achieved on benchmark datasets over the state-of-the-art methods.