Revisiting Support Vector Machines: A Distance-Based Alternative to Deep Learning for Efficient Text Classification

Support Vector Machines (SVMs) remain competitive in text classification, sometimes achieving comparable performance with the deep learning approach, due to their strong generalization ability and robustness to overfitting. However, their strong classification performance relies heavily on the selection of kernel functions and parameters, resulting in substantial hyperparameter tuning. A previous study has proposed Euclidean-SVM, which modifies the SVM decision mechanism by replacing the optimal separating hyperplane with a distance-based decision rule. This proposed approach reported reduced dependency on kernel functions and regularization parameters, resulting in robust performance with lower sensitivity to hyperparameter changes. Nevertheless, Euclidean-SVM only investigates Euclidean distance; other distance metrics that may achieve comparable performance remain unexplored. This study aims to evaluate the effectiveness of multiple distance metrics as an alternative decision function in the distance-based SVM framework for text classification. The distances, including Euclidean distance, Manhattan distance, Chebyshev distance, Cosine distance, and Minkowski distance, were investigated. The experimental results demonstrate that Euclidean and Cosine distances achieve stable and competitive classification performance across a wide range of hyperparameter configurations, reaching an accuracy of approximately 84-97% across the evaluated datasets. In contrast, the remaining distances, including Manhattan, Chebyshev, and Minkowski, exhibit significantly lower performance, reaching an accuracy between 14 and 71%, indicating the discriminative power of these distances is lower. A preliminary comparison with the deep learning model Long Short-Term Memory (LSTM) further shows that the distance-based SVM, including Euclidean and Cosine-based SVM, achieves higher performance and greater stability. These findings suggest that Euclidean and Cosine distances enhance the robustness of SVM-based text classification, while reducing the need for extensive hyperparameter tuning, making them suitable for resource-constrained environments compared to deep learning.