BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling
Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, Anton van den Hengel
Abstract
Inferring the 3D structure of a non-rigid dynamic scene from a single moving camera is an under-constrained problem. Inspired by the remarkable progress of neural radiance fields (NeRFs) in photo-realistic novel view synthesis of static scenes, it has also been extended to dynamic settings. Such methods heavily rely on implicit neural priors to regularize the problem. In this work, we take a step back and investigate how current implementations may entail deleterious effects including limited expressiveness, entanglement of light and density fields, and sub-optimal motion localization. Further, we devise a factorisation-based framework that represents the scene as a composition of bandlimited, high-dimensional signals. We demonstrate compelling results across complex dynamic scenes that involve changes in lighting, texture and long-range dynamics.
BibTeX
@article{Ramasinghe_Shevchenko_Avraham_van den Hengel_2024, title={BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling}, volume={38}, url={https://ojs.aaai.org/index.php/AAAI/article/view/28264}, DOI={10.1609/aaai.v38i5.28264}, abstractNote={Inferring the 3D structure of a non-rigid dynamic scene from a single moving camera is an under-constrained problem. Inspired by the remarkable progress of neural radiance fields (NeRFs) in photo-realistic novel view synthesis of static scenes, it has also been extended to dynamic settings. Such methods heavily rely on implicit neural priors to regularize the problem. In this work, we take a step back and investigate how current implementations may entail deleterious effects including limited expressiveness, entanglement of light and density fields, and sub-optimal motion localization. Further, we devise a factorisation-based framework that represents the scene as a composition of bandlimited, high-dimensional signals. We demonstrate compelling results across complex dynamic scenes that involve changes in lighting, texture and long-range dynamics.}, number={5}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Ramasinghe, Sameera and Shevchenko, Violetta and Avraham, Gil and van den Hengel, Anton}, year={2024}, month={Mar.}, pages={4641-4649} }