Hybrid Domain Learning towards Light Field Spatial Super-Resolution using Heterogeneous Imaging

Abstract

Light field (LF) cameras usually capture dense angular samples, but suffer from low spatial resolution. Existing single-LF super-resolution methods struggle with textures at larger scales (e.g., 8×). To address this issue, this paper proposes a novel hybrid domain learning-based method to enhance LF spatial resolution from heterogeneous imaging (integrating an LF camera and a 2D digital camera). The proposed method consists of two core modules, namely LF feature alignment module and cross-domain multi-scale fusion module. The former combines optical flow and deformable convolution to gradually align the 2D high-resolution features with the low-resolution LF features. The latter progressively fuses the aligned multi-resolution LF features to enable high-quality reconstruction. Experimental results show the proposed method recovers fine textures and preserves accurate angular consistency, and outperforms the state-of-the-art methods in both quantitative and qualitative comparisons.