Open AccessHyperspectral imaging from a raw mosaic image with end-to-end learning
Author(s) -
Hao Fu,
Liheng Bian,
Xianbin Cao,
Jun Zhang
Publication year - 2020
Publication title -
optics express
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.394
H-Index - 271
ISSN - 1094-4087
DOI - 10.1364/oe.372746
Subject(s) - hyperspectral imaging , computer science , artificial intelligence , rgb color model , computer vision , image resolution , full spectral imaging , noise (video) , iterative reconstruction , pattern recognition (psychology) , remote sensing , image (mathematics) , geology
Hyperspectral imaging provides rich spatial-spectral-temporal information with wide applications. However, most of the existing hyperspectral imaging systems require light splitting/filtering devices for spectral modulation, making the system complex and expensive, and sacrifice spatial or temporal resolution. In this paper, we report an end-to-end deep learning method to reconstruct hyperspectral images directly from a raw mosaic image. It saves the separate demosaicing process required by other methods, which reconstructs the full-resolution RGB data from the raw mosaic image. This reduces computational complexity and accumulative error. Three different networks were designed based on the state-of-the-art models in literature, including the residual network, the multiscale network and the parallel-multiscale network. They were trained and tested on public hyperspectral image datasets. Benefiting from the parallel propagation and information fusion of different-resolution feature maps, the parallel-multiscale network performs best among the three networks, with the average peak signal-to-noise ratio achieving 46.83dB. The reported method can be directly integrated to boost an RGB camera for hyperspectral imaging.