Wavelength encoding spectral imaging based on the combination of deeply learned filters and an RGB camera

Hyperspectral imaging is a critical tool for gathering spatial-spectral information in various scientific research fields. As a result of improvements in spectral reconstruction algorithms, significant progress has been made in reconstructing hyperspectral images from commonly acquired RGB images. However, due to the limited input, reconstructing spectral information from RGB images is ill-posed. Furthermore, conventional camera color filter arrays (CFA) are designed for human perception and are not optimal for spectral reconstruction. To increase the diversity of wavelength encoding, we propose to place broadband encoding filters in front of the RGB camera. In this condition, the spectral sensitivity of the imaging system is determined by the filters and the camera itself. To achieve an optimal encoding scheme, we use an end-to-end optimization framework to automatically design the filters’ transmittance functions and optimize the weights of the spectral reconstruction network. Simulation experiments show that our proposed spectral reconstruction network has excellent spectral mapping capabilities. Additionally, our novel joint wavelength encoding imaging framework is superior to traditional RGB imaging systems. We develop the deeply learned filter and conduct actual shooting experiments. The spectral reconstruction results have an attractive spatial resolution and spectral accuracy.