The scalability of predictive models has become a critical factor in modern machine learning, as data volumes grow and computational resources diversify. This study presents an empirical benchmark of three widely used regression paradigms: Elastic Net, XGBoost, and Multi-Layer Perceptrons (MLPs). The Obesity Estimation dataset is used to evaluate both predictive performance and computational scalability across multi-core CPUs and GPUs. Unlike prior studies that primarily emphasize accuracy, we explicitly examine the trade-offs between accuracy, training time, and hardware efficiency. Models are evaluated under staged training loads (10–100% of data) with grid-searched hyperparameters (for Elastic Net and XGBoost) and regularized deep architectures (for MLP). Results demonstrate that while XGBoost achieves the highest predictive accuracy (R2 = 0.91), it incurs significant computational overhead on CPUs, whereas GPU acceleration substantially improves its scalability. MLPs provide competitive accuracy (R2 = 0.87) with an order-of-magnitude lower training time on GPUs, making them attractive for rapid or repeated retraining. Elastic Net offers interpretability and linear scalability on CPUs, but lags in predictive power. These findings provide practitioners with a decision framework: XGBoost for maximum accuracy, MLPs for efficient retraining, and Elastic Net for interpretability and small-scale tasks. More broadly, this work highlights that hardware selection is as important as algorithm choice, with GPUs serving as enablers of state-of-the-art performance on structured data.
Mahmood et al. (Thu,) studied this question.