Region-aware RGB and near-infrared image fusion

This paper proposes a region-aware fusion method, called RaIF, for RGB and near-infrared (NIR) outdoor scenery image fusion. The method is motivated by the observation that current fusion approaches produce gray appearance in overexposed sky regions and distortion in vegetation regions. RaIF generates the region probability maps by exploiting their specific characteristics in the visible and NIR spectra. It recovers the overexposed sky regions by employing the intrinsic channel correlation between RGB and NIR images, and enhances the vegetation regions in an adjustable manner. RaIF formulates image fusion problem as a gradient-domain optimization problem with luminance and chromaticity regularizations. Experimental results validate the superiority of RaIF that produces fused images with improved appearance in the sky and vegetation regions, and achieves the state-of-the-art performance quantitatively and qualitatively. Furthermore, RaIF can act as a refinement module that improves the fusion results of current deep learning based approaches. It is also capable of recovering specular highlight regions other than sky overexposure.