ABSTRACT This review provides a comprehensive Systematic Literature Review (SLR) and Systematic Mapping Study (SMS) of regression research published in Wiley‐indexed venues between 1975 and 2025 . Analyzing a validated corpus of 1466 publications—of which 1174 dated studies form the empirical base—this study traces the theoretical, methodological, and epistemic evolution of regression modeling. The findings reveal a progressive convergence between classical penalized methods (e.g., LASSO, Elastic Net), Bayesian shrinkage approaches, and machine learning‐based regressors such as ensembles and neural networks. From 2022 onward, a distinct interpretability‐driven era emerges, marked by the selective uptake of post hoc explanation techniques like SHAP and LIME. Despite their conceptual appeal, these tools remain underutilized in the corpus—due in part to computational costs, concerns over stability, and the prioritization of embedded interpretability strategies. Through thematic synthesis, the review identifies persistent trade‐offs in regression design—bias versus variance, sparsity versus signal retention, and accuracy versus interpretability—and highlights exemplar hybrid approaches that balance these tensions. The study concludes with a research agenda targeting five frontiers: (1) integrated interpretability systems, (2) unified theoretical modeling frameworks, (3) scalable regression for high‐dimensional and streaming data, (4) robust and uncertainty‐aware modeling, and (5) fairness and transparency in applied settings. Together, these contributions reposition regression as a unified, interdisciplinary field grounded in statistical insight, computational design, and ethical accountability.
Bendersky et al. (Sun,) studied this question.