A comparative study of Convolutional Neural Network and Vision Transformer models as classifiers of east Asian traditional clothing

Convolutional Neural Network (CNN) and Vision Transformer (ViT) are two models that have been at the center of discussion in the field of machine learning due to their accuracy in classifying images. Current research has shown that different image classification tasks lead to different performances for both models. One area of research that is still under-explored when comparing CNN and ViT is the task of classifying images of traditional clothing from various countries. In this study, a CNN and a ViT were constructed and trained on a dataset of Chinese, South Korean, and Japanese traditional clothing images. The two models were then quantitatively compared across different aspects of accuracy utilizing four evaluation metrics: accuracy, precision, recall, and F1 score. The results showed that the ViT–with an average score of 88%–outperformed the CNN–which had an average score of 73.15%–in every evaluation metric, underscoring the potential of the ViT for classifying traditional clothing. Additionally, the models were deliberately fed out-of-dataset data to find if there were patterns in the resulting obvious mis-classification of such images. Analysis of such patterns revealed features and attributes that the models used to classify data, which unlocked model explainability; the current Achilles’ heel for stand-alone model adoption. Overall, these findings can prove useful to individuals in the field of cultural preservation by showcasing that the ViT can be used as a tool to accurately classify traditional clothing attire from China, Japan and Korea.