Cross-Spectral Gaussian Splatting with Spatial Occupancy Consistency
Haipeng Guo, Huanyu Liu, Jiazheng Wen, Junbao Li
Abstract
Using images captured by cameras with different light spectrum sensitivities, training a unified model for cross-spectral scene representation is challenging. Recent advances have shown the possibility of jointly optimizing cross-spectral relative poses and neural radiance fields using normalized cross-device coordinates. However, such method suffers from cross-spectral misalignment when collecting data asynchronously from devices and lacks the capability to render in real-time or handle large scenes. We address these issues by proposing cross-spectral Gaussian Splatting with spatial occupancy consistency, strictly aligns cross-spectral scene representation by sharing explicit Gaussian surfaces across spectra and separately optimizing each view's extrinsic using a matching-optimizing pose estimation method. Additionally, to address field-of-view differences in cross-spectral cameras, we improve the adaptive densify controller to fill non-overlapping areas. Comprehensive experiments demonstrate that SOC-GS achieves superior performance in novel view synthesis and real-time cross-spectral rendering.
BibTeX
@article{Guo_Liu_Wen_Li_2025, title={Cross-Spectral Gaussian Splatting with Spatial Occupancy Consistency}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/32333}, DOI={10.1609/aaai.v39i3.32333}, abstractNote={Using images captured by cameras with different light spectrum sensitivities, training a unified model for cross-spectral scene representation is challenging. Recent advances have shown the possibility of jointly optimizing cross-spectral relative poses and neural radiance fields using normalized cross-device coordinates. However, such method suffers from cross-spectral misalignment when collecting data asynchronously from devices and lacks the capability to render in real-time or handle large scenes. We address these issues by proposing cross-spectral Gaussian Splatting with spatial occupancy consistency, strictly aligns cross-spectral scene representation by sharing explicit Gaussian surfaces across spectra and separately optimizing each view’s extrinsic using a matching-optimizing pose estimation method. Additionally, to address field-of-view differences in cross-spectral cameras, we improve the adaptive densify controller to fill non-overlapping areas. Comprehensive experiments demonstrate that SOC-GS achieves superior performance in novel view synthesis and real-time cross-spectral rendering.}, number={3}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Guo, Haipeng and Liu, Huanyu and Wen, Jiazheng and Li, Junbao}, year={2025}, month={Apr.}, pages={3229-3237} }