NeRF-MS: Neural Radiance Fields with Multi-Sequence
Peihao Li, Shaohui Wang, Chen Yang, Bingbing Liu, Weichao Qiu, Haoqian Wang
Abstract
Neural radiance fields (NeRF) achieve impressive performance in novel view synthesis when trained on only single sequence data. However, leveraging multiple sequences captured by different cameras at different times is essential for better reconstruction performance. Multi-sequence data takes two main challenges: appearance variation due to different lighting conditions and non-static objects like pedestrians. To address these issues, we propose NeRF-MS, a novel approach to training NeRF with multi-sequence data. Specifically, we utilize a triplet loss to regularize the distribution of per-image appearance code, which leads to better high-frequency texture and consistent appearance, such as specular reflections. Then, we explicitly model non-static objects to reduce floaters. Extensive results demonstrate that NeRF-MS not only outperforms state-of-the-art view synthesis methods on outdoor and synthetic scenes, but also achieves 3D consistent rendering and robust appearance controlling. Project page: https://nerf-ms.github.io/.
BibTeX
@inproceedings{iccv2023_nerfmsneuralradi,
title = {NeRF-MS: Neural Radiance Fields with Multi-Sequence},
author = {Peihao Li and Shaohui Wang and Chen Yang and Bingbing Liu and Weichao Qiu and Haoqian Wang},
booktitle = {ICCV 2023},
year = {2023}
}