Open AccessData-driven framework for high-accuracy color restoration of RGBN multispectral filter array sensors under extremely low-light conditions
Author(s) -
Yanan Cao,
Bowen Zhao,
Xin Tong,
Jian Chen,
Jiangxin Yang,
Xin Li
Publication year - 2021
Publication title -
optics express
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.394
H-Index - 271
ISSN - 1094-4087
DOI - 10.1364/oe.426940
Subject(s) - multispectral image , rgb color model , computer science , artificial intelligence , computer vision , filter (signal processing) , color filter array , convolutional neural network , optical filter , image restoration , optics , color gel , image processing , materials science , physics , image (mathematics) , layer (electronics) , composite material , thin film transistor
RGBN multispectral filter array provides a cost-effective and one-shot acquisition solution to capture well-aligned RGB and near-infrared (NIR) images which are useful for various optical applications. However, signal responses of the R, G, B channels are inevitably distorted by the undesirable spectral crosstalk of the NIR bands, thus the captured RGB images are adversely desaturated. In this paper, we present a data-driven framework for effective spectral crosstalk compensation of RGBN multispectral filter array sensors. We set up a multispectral image acquisition system to capture RGB and NIR image pairs under various illuminations which are subsequently utilized to train a multi-task convolutional neural network (CNN) architecture to perform simultaneous noise reduction and color restoration. Moreover, we present a technique for generating high-quality reference images and a task-specific joint loss function to facilitate the training of the proposed CNN model. Experimental results demonstrate the effectiveness of the proposed method, outperforming the state-of-the-art color restoration solutions and achieving more accurate color restoration results for desaturated and noisy RGB images captured under extremely low-light conditions.