Spatial-Spectral Deep Learning for Prostate Cancer Tissue Classification in Infrared Spectroscopy

Modern methods of infrared (IR) spectroscopy yield full IR absorbance spectra in arrays, forming hyperspectral images. End-to-end processing of these images via deep learning seems ideal for exploiting their high dimensionality and wealth of spatial and spectral information, but recent research suggests that convolution-based architectures may have a spatial bias. Toward the goal of improved prostate cancer tissue classification, we compare a variety of deep learning classifiers for IR spectroscopy and probe the impact of a bottleneck which compresses the spectral dimension. We find a strong correlation between model spatial receptive field and classification performance, with the highest performance achieved by a modified Vision Transformer model. Conversely, we find only limited correlation between spectral information and deep learning model performance: we find that a spectral bottleneck of just 16 features has only a negligible effect on all neural network models, including convolution-eschewing transformer architectures and a multilayer perceptron model utilizing no spatial information. Rather than any particular network component inducing a spatial bias, the breadth of architectures exhibiting little dependence on spectral information implies that tissue classification itself is characterized by only a small set of spectral features. This, in turn, suggests that success at tissue classification may be a poor benchmark in the development of deep learning models designed to effectively utilize the spectral dimension.