Exploiting full sparsity: double adaptive sieving for large-scale and high dimensional support vector machines

The escalating scale and dimensionality of modern datasets present significant computational hurdles for traditional machine learning tools, for which efficiency and accuracy are both required. This paper tackles the computational bottleneck faced by Support Vector Machines (SVMs) when applied to large-scale, high-dimensional data. We propose a Double Adaptive Sieving (DAS) framework that concurrently leverages the inherent sparsity structures in both samples and features. This approach efficiently constructs a sparse solution path, achieving effective feature selection while maintaining high classification accuracy. The inner optimization problem is solved through a sparse Newton-based method that exploits sparse second-order information to achieve a superlinear convergence rate. Furthermore, we establish theoretical guarantees regarding the finite termination property of the proposed framework. Comprehensive numerical experiments demonstrate the superior efficiency of the DAS framework.