Multimodal Retinal Image Registration and Fusion Based on Sparse Regularization via a Generalized Minimax-concave Penalty

Abstract

We introduce a novel framework for the fusion of retinal OCT and confocal images of mice with uveitis. Input images are semi-automatically registered and then fused to provide more informative retinal images for analysis by ophthalmologists and clinicians. The proposed feature-based registration approach extracts vessels through the use of the ISO-DATA algorithm and morphological operations, in order to match confocal images with OCT images. Image fusion is formulated as an inverse problem, with the corresponding cost function containing two data attachment terms and a non-convex penalty function (the Generalized Minimax-Concave function) that maintains the overall convexity of the problem. The minimization of the cost function is thus tackled by convex optimization. Objective assessment results on image fusion show that this novel image fusion method has competitive performance when compared to existing image fusion methods. Some features of retina that cannot be observed directly in the original images are shown to be enhanced in the fused representations.