Optimal Selection of Parameter‐Wise Shrinkage Parameters in Penalized Regression Models

Standard penalized regression methods typically employ no more than two tuning parameters due to computational constraints. However, when covariate effects differ in magnitude or structure, allowing variable‐specific regularization can lead to improved estimation and prediction. This paper proposes a flexible penalized regression framework that assigns a distinct penalty parameter to each covariate. To mitigate the computational burden associated with tuning a penalty vector, we adopt a mean squared error criterion guided by an initial estimator, which may require standard tuning procedures. The methodology is developed for both mean and quantile regression models under a unified framework. Simulation studies and real data analyses demonstrate that the proposed approach yields improved estimation accuracy and shrinkage performance compared to existing methods.