Regression analysis is a core analytical tool widely employed across diverse domains for predicting continuous outcomes, serving as a cornerstone of statistical inference and machine learning applications ranging from economic trend forecasting to healthcare risk assessment and real estate valuation. Choosing an effective regression technique is critical for accurate predictions, yet a daunting challenge for non-experts due to the wide variety of methods, each with distinct assumptions, tuning requirements and applicability boundaries. To address this dilemma, this study conducts a rigorous empirical comparison of five popular regression techniques—Ordinary Least Squares (OLS), Ridge regression, Lasso regression, Elastic Net, and Polynomial regression—applied to house price prediction using two benchmark datasets: the classic Boston Housing dataset and the comprehensive California Housing dataset. A multi-dimensional evaluation framework was adopted, including quantitative metrics (Mean Squared Error (MSE) and coefficient of determination () and qualitative diagnostics (residual analysis and Quantile-Quantile (QQ) plots) to assess prediction accuracy and error distribution. Results indicate that Polynomial regression consistently achieves superior performance across both datasets, highlighting its effectiveness in capturing the complex nonlinear relationships inherent in housing data. Ridge, Lasso, and Elastic Net provide comparable but lower performance, with strengths in mitigating multicollinearity rather than enhancing nonlinear fitting. OLS yields acceptable baseline results but less robust performance when confronted with real-world nonlinearities. These findings offer clear practical guidance for non-experts seeking reliable “out-of-the-box” regression techniques, and contribute valuable insights to assist practitioners in model selection for real-world predictive tasks without extensive tuning.
Wang et al. (Fri,) studied this question.