Adaptive Large-Scale Novel View Image Synthesis for Autonomous Driving Datasets

Abstract

Novel view image synthesis for large-scale outdoor traffic scenes presents significant challenges, including inaccurate depth measurements, moving objects, wide-angle rendering requirements, and the increased demand for memory and computational resources. In this paper, we propose an adaptive pipeline that constructs high-fidelity 3D surfel models and synthesizes realistic novel views in real time. Our contributions are threefold: 1) developing depth-refinement and moving-object-removal techniques to robustly reconstruct surfel-based scene geometry, while minimizing computational overhead; 2) developing a self-adaptive rendering mechanism which adjusts surfel geometry for large-scale scenes within constrained memory; 3) developing a hyper-parameter tuning approach for optimal surfel construction and rendering performance. An optional GAN-based inpainting module fills missing backgrounds (e.g., sky). Experiments on the KITTI dataset and CARLA simulator show that our method achieves image quality comparable to SOTA NeRF and 3D Gaussian Splatting techniques with significantly improved computational efficiency. This makes our approach particularly well-suited for large-scale traffic scenarios. Our simulation datasets with ground-truth data and source code are available at https://github.com/Billy1203/SurfelMapping.