Lightweight ViT Model for Textile Material Classification

This study proposes a lightweight vision transformer (ViT) architecture for textile material classification subject to on-device computational constraints. Although ViTs effectively model global contextual relationships through self-attention, their high computational complexity - especially from multihead self-attention (MHSA) limits real-time deployment in resource-constrained environments. Given that textile images are dominated by repetitive and self-similar texture patterns, we investigate herein whether simplified attention mechanisms can achieve comparable performance with lower complexity. We design lightweight ViT variants by comparing MHSA and single-head self-attention (SHSA), incorporating a convolution-based feature-extraction (FE) block before the transformer encoder to enhance texture representation and reduce token length, and analyzing the effects of transformer depth and embedding dimension under similar computational budget conditions. Experimental results show that FE-based ViT models outperform convolutional- neural-network-based baselines at lower computational cost. In particular, the SHSA-FE model achieves classification accuracy outcomes up to 99% using only 0.23 GFLOPs while maintaining high accuracy across architectural variations. Attention map analysis further demonstrates that simplified attention produces more structurally aligned responses to repetitive texture patterns. These results suggest that for texture-dominant tasks, such as textile material classification, structurally simplified attention combined with efficient feature extraction provides a more suitable and computationally efficient alternative to conventional multihead transformer architectures for on-device applications.