HyperReel: High-Fidelity 6-DoF Video With Ray-Conditioned Sampling
Benjamin Attal, Jia-Bin Huang, Christian Richardt, Michael Zollhöfer, Johannes Kopf, Matthew O’Toole, Changil Kim
Abstract
Volumetric scene representations enable photorealistic view synthesis for static scenes and form the basis of several existing 6-DoF video techniques. However, the volume rendering procedures that drive these representations necessitate careful trade-offs in terms of quality, rendering speed, and memory efficiency. In particular, existing methods fail to simultaneously achieve real-time performance, small memory footprint, and high-quality rendering for challenging real-world scenes. To address these issues, we present HyperReel --- a novel 6-DoF video representation. The two core components of HyperReel are: (1) a ray-conditioned sample prediction network that enables high-fidelity, high frame rate rendering at high resolutions and (2) a compact and memory-efficient dynamic volume representation. Our 6-DoF video pipeline achieves the best performance compared to prior and contemporary approaches in terms of visual quality with small memory requirements, while also rendering at up to 18 frames-per-second at megapixel resolution without any custom CUDA code.
BibTeX
@inproceedings{cvpr2023_hyperreelhighfid,
title = {HyperReel: High-Fidelity 6-DoF Video With Ray-Conditioned Sampling},
author = {Benjamin Attal and Jia-Bin Huang and Christian Richardt and Michael Zollhöfer and Johannes Kopf and Matthew O’Toole and Changil Kim},
booktitle = {CVPR 2023},
year = {2023}
}