Incorporating Symmetric Smooth Regularizations into Sparse Logistic Regression for Classification and Feature Extraction

This paper introduces logistic regression with sparse and smooth regularizations (LR-SS), a novel framework that simultaneously enhances both classification and feature extraction capabilities of standard logistic regression. By incorporating a family of symmetric smoothness constraints into sparse logistic regression, LR-SS uniquely preserves underlying structures inherent in structured data, distinguishing it from existing approaches. Within the minorization–maximization (MM) framework, we develop an efficient optimization algorithm that combines coordinate descent with soft-thresholding techniques. Through extensive experiments on both simulated and real-world datasets, including time series and image data, we demonstrate that LR-SS significantly outperforms conventional sparse logistic regression in classification tasks while providing more interpretable feature extraction. The results highlight LR-SS’s ability to leverage sparse and symmetric smooth regularizations for capturing intrinsic data structures, making it particularly valuable for machine learning applications requiring both predictive accuracy and model interpretability.