Pyramid Spatial Feature Transform and Shared-Offsets Deformable Alignment Based Convolutional Network for HDR Imaging

Abstract

To generate ghost-free high dynamic range (HDR) images by merging multiple differently exposed low dynamic range (LDR) images, the key is to handle ill-exposed areas in the input LDR images and misalignment among them. In this paper, a Pyramid Spatial Feature Transform and shared-offsets Deformable convolutional Network (PSFTDNet) is proposed to achieve this target. The pyramid spatial feature transform module tackles ill-exposed areas, which modulates the features to exploit complementary information of them in a coarse-to-fine manner. The shared-offsets deformable alignment handles misalignment among the input images, which applies the offsets used to align exposure-aligned images to align all the features. Experiments on the NTIRE HDR challenge dataset and Kalantari dataset show that the proposed PSFTDNet outperforms all the conventional methods with PSNR-L scores of 42.31 dB and 41.54 dB, and PSNR-T scores of 34.78 dB and 43.56 dB.