S3ViT: self-supervised spectral vision transformer framework for hyperspectral unmixing

Hyperspectral unmixing aims to decompose each pixel in a hyperspectral image into a set of constituent endmembers and their corresponding abundances. Recent deep learning based approaches have demonstrated strong performance in capturing both spectral and spatial features. However, obtaining reliable per-pixel abundance ground truth in real hyperspectral scenes is generally infeasible, which motivates unsupervised and self-supervised unmixing strategies. In this work, we propose S 3 ViT, a self-supervised Spectral Vision Transformer designed for pixel-wise hyperspectral unmixing. The transformer captures spectral and spatial dependencies by applying self-attention over the full sequence of pixel tokens ( 1 × 1 ) augmented with learnable positional embeddings. It operates without ground-truth annotations by generating pseudo labels through an unsupervised process: first, Singular Value Decomposition (SVD) is used to estimate the number of endmembers based on a thresholded singular value spectrum; then, k-means clustering provides cluster-derived priors that are used to form a contextual token and initialize spectral prototypes, without being treated as true abundance supervision. To guide training, two initialization tokens, one from Vertex Component Analysis (VCA) and one from the k-means cluster-derived priors, are embedded alongside patch tokens into the transformer. The model learns to estimate abundance maps while enforcing the Abundance Non-negativity and Sum-to-One constraints through ReLU and Softmax layers. Endmember spectra are later estimated from pixels with high predicted abundances. We evaluated S 3 ViT on the Samson, Jasper Ridge and Washington DC Mall benchmark datasets and compared it to state-of-the-art geometrical and deep learning methods. Our model achieves superior or comparable performance in both RMSE and SAD metrics, with up to 31% improvement in SAD and 25% in RMSE. These results indicate that a compact pixel-token ViT, guided by weak spectral priors and optimized via reconstruction losses, can achieve competitive unmixing performance on standard benchmarks.