Deep Cost Ray Fusion for Sparse Depth Video Completion

Jungeon Kim; Soongjin Kim; Jaesik Park; Seungyong Lee*

ECCV 2024poster0 citations

Deep Cost Ray Fusion for Sparse Depth Video Completion

Jungeon Kim, Soongjin Kim, Jaesik Park, Seungyong Lee*

Abstract

"In this paper, we present a learning-based framework for sparse depth video completion. Given a sparse depth map and a color image at a certain viewpoint, our approach makes a cost volume that is constructed on depth hypothesis planes. To effectively fuse sequential cost volumes of the multiple viewpoints for improved depth completion, we introduce a learning-based cost volume fusion framework, namely RayFusion, that effectively leverages the attention mechanism for each pair of overlapped rays in adjacent cost volumes. As a result of leveraging feature statistics accumulated over time, our proposed framework consistently outperforms or rivals state-of-the-art approaches on diverse indoor and outdoor datasets, including the KITTI Depth Completion benchmark, VOID Depth Completion benchmark, and ScanNetV2 dataset, using much fewer network parameters."

BibTeX

@inproceedings{eccv2024_deepcostrayfusio,
  title = {Deep Cost Ray Fusion for Sparse Depth Video Completion},
  author = {Jungeon Kim and Soongjin Kim and Jaesik Park and Seungyong Lee*},
  booktitle = {ECCV 2024},
  year = {2024}
}