Yuanwen Yue*

Improving 2D Feature Representations by 3D-Aware Fine-Tuning

"Current visual foundation models are trained purely on unstructured 2D data, limiting their understanding of 3D structure of objects and scenes. In this work, we show that fine-tuning on 3D-aware data improves the quality of emerging semantic features. We design a method to lift semantic 2D feature…