Snapshot multispectral imaging using optical-informed learning

This is the authors' abstract. We don't add key points for this paper.

Multispectral imaging has wide applications in the fields of science and engineering, as it offers more comprehensive information than RGB data, which is particularly useful in addressing issues such as metamerism. However, traditional multispectral imaging is limited by factors such as time, space, and accuracy, which hinder its ability to achieve fast, precise, and cost-effective spectral imaging. In this paper, an optical-informed deep learned multispectral imaging technique is proposed to achieve accurate, fast, and plug-and-play multispectral imaging. By modeling the spectral estimation as an inverse problem-solving task, an end-to-end neural network comprising mixture attention modules is specifically designed for automatic transformation from a one-shot RGB image to a hyperspectral image, incorporating optical priors to improve network performance and its interpretability. A pilot optical system comprising a complex illumination simulation lightbox and a beamsplitter is established to validate the effectiveness under different illumination conditions. The experimental results indicate that the proposed technique achieves high spectral reconstruction accuracy, with an MSE of 0.00426 and an SSIM of 0.942, representing a 29% improvement in MSE compared to HSCNN + . Experiments under different lighting conditions and response curves are conducted to ensure robustness in all scenarios. The pipeline achieves real-time and robust multispectral imaging based on a one-shot RGB image, providing a new panel for snapshot multispectral imaging, with the potential for wide application in medical imaging, quality monitoring, and mineral exploration.