DU-PMVS: Learned Patchmatch Multi-View Stereo Based on Deformable Feature Pyramid and Uncertainty Awareness Modeling

Abstract

Multi-View Stereo is widely utilized for reconstructing the dense geometric structure of objects from multiple viewpoints. Recently, learning-based PatchMatch MVS methods have attracted significant attention due to their high efficiency and accuracy. However, existing methods neglect the constraints of convolutional kernel receptive fields and uniform depth sampling. In this paper, we propose a novel PatchMatch MVS method named DU-PMVS, which concurrently supports adaptive feature extraction and depth sampling. Specifically, we design a Deformable Feature Pyramid Extractor to capture multi-scale features for each pixel, thereby enhancing the representation of contextual information. Additionally, we propose Uncertainty Probability Distribution-guided Depth Modeling, which addresses the issue of cumulative errors in coarse-to-fine structure by exploring richer uncertainty distributions and generating more effective depth hypotheses. Experimental results on the DTU and Tanks & Temples datasets demonstrate that our DU-PMVS can reconstruct more completed point clouds with low memory, particularly in challenging regions with weak textures.