Multi-channel hyperspectral image feature fusion with CBAM and improved transformer encoder for mulberry leaf moisture content and variety identification

Accurate moisture content determination and varietal classification are essential for the diversified utilization of mulberry leaves. This study employed hyperspectral imaging (937–1718 nm) from six mulberry varieties to develop robust regression and classification models. After extracting average spectral and textural features, conventional partial least squares regression and back-propagation neural networks were established, confirming the superiority of spectral over textural features. To enhance modeling performance, one-dimensional spectra were converted into four two-dimensional representations: band interaction ratio (BIR), Gramian angular difference field (GADF), unthresholded recurrence plot (UTRP), Markov transition field (MTF), moving window wavelet transform (MWWT), and continuous wavelet transform. These representations, along with raw spectra and hyperspectral cubes, were used to construct various convolutional neural network (CNN) models, by comprehensively exploiting the spatial features and the multi-band synergistic interactions associated with key chemical constituents in mulberry leaves, such as moisture, protein, crude fiber, polysaccharides, flavonoids, and alkaloids. Results showed that CNNs based on BIR, GADF, UTRP, and MWWT significantly improved moisture prediction, while BIR, MWWT, and the grayscale image of mulberry leaf excelled in varietal classification. By integrating multiple 2D features into a multi-channel CNN enhanced with a convolutional block attention module and an optimized Transformer encoder, prediction accuracy was further improved. The optimized model achieved a determination coefficient of 0.972 in moisture prediction and a classification accuracy of 98.571% for varietal identification. This work demonstrates that fusing multi-source hyperspectral representations with deep learning effectively advances quality assessment for industrial crops. • Multi-channel fusion framework to detect mulberry leaves quality is proposed. • Spectral transformations, BIR, GADF, UTRP, MWWT, enhance deep feature representation. • Modified Transformer Encoder module optimally leverages inter-channel synergies. • State-of-the-art accuracy is achieved for moisture content and variety identification.