Comparative Study of Regression Models for Continuous Function Approximation

Regression models are widely used for continuous function approximation in applied research, yet selecting an appropriate model remains challenging for applied users who must balance predictive accuracy, interpretability, robustness, computational cost, and preprocessing requirements. This methodological review provides a decision-oriented synthesis of regression model families, preprocessing strategies, and evaluation criteria for transparent and reproducible model selection. The reviewed methods are organized by modeling principle, including linear and regularized models, robust and distribution-aware estimators, online learning methods, tree-based ensembles, kernel-based and probabilistic approaches, instance-based regressors, neural networks, and symbolic regression. The main contribution is a practical framework that connects data characteristics, including linearity, dimensionality, feature scale, target distribution, noise, outliers, and sample size, with suitable model families, preprocessing choices, and performance metrics. The review distinguishes theoretical guarantees, empirical tendencies, and implementation-dependent behavior because properties such as robustness, interpretability, scalability, and approximation capacity cannot be reduced to universal binary categories. The resulting comparative tables and decision criteria provide a compact reference for applied researchers designing regression workflows that are theoretically grounded, practically feasible, and aligned with research objectives.