PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models

Abstract

The development of 3D Vision-Language Models (VLMs), crucial for applications in robotics, autonomous driving, and augmented reality, is severely constrained by the scarcity of paired 3D-text data. Existing methods rely solely on next-token prediction loss, using only language tokens for supervision. This results in inefficient utilization of limited 3D data and leads to a significant degradation and loss of valuable geometric information in intermediate representations.To address these limitations, we propose \mname , a novel feature-level alignment regularization method. \mname explicitly supervises intermediate point cloud representations to preserve fine-grained 3D geometric-semantic information throughout the language modeling process. Specifically, we constrain the intermediate point cloud tokens within the LLM to align with visual input tokens via a consistency loss. By training only a lightweight alignment projector and LoRA adapters, \mname achieves explicit feature-level supervision with minimal computational overhead, effectively preventing geometric degradation.Extensive experiments on ModelNet40 and Objaverse datasets demonstrate that our method achieves 2.08 pp improvement on average for classification tasks, with a substantial 7.50 pp gain on the challenging open-vocabulary Objaverse classification task and 4.88 pp improvement on 3D object captioning evaluated by Qwen2-72B-Instruct, validating the effectiveness of \mname .