These Magic Moments: Differentiable Uncertainty Quantification of Radiance Field Models

Abstract

Uncertainty quantification is crucial for autonomous systems, enabling safe and robust decision making in tasks ranging from active perception to robotic planning. This paper introduces a novel approach to quantify uncertainty for radiance fields by deriving pixel-wise moment expressions from the rendering equation. While radiance fields offer powerful scene representations, their high dimensionality and complexity have historically made uncertainty quantification computationally prohibitive for real-time applications. This paper demonstrates that the probabilistic nature of the rendering process enables efficient and differentiable computation of higher-order moments for radiance field outputs, including color, depth, and semantic predictions. The proposed method outperforms existing radiance field uncertainty estimation techniques while offering a more direct, computationally efficient, and differentiable formulation without the need for post-processing. Beyond uncertainty quantification, this paper also illustrates the utility of the proposed approach in downstream applications such as next-best-view (NBV) selection and active ray sampling for neural radiance field training. Extensive experiments on both synthetic and real-world scenes demonstrate state-of-the-art performance, confirming that principled uncertainty quantification can be seamlessly integrated into radiance field pipelines without sacrificing efficiency or accuracy.