Revisiting training-free tabular models: A robustness and efficiency study

Training-free classifiers for tabular data eliminate dataset-specific model fitting by pre-training a meta-learner that generalises to new tasks without gradient updates. HyperFast Bonet et al. (2024), introduced at AAAI 2024, generates task-specific classifier weights through a hypernetwork and is promoted as suitable for resource-constrained deployment. However, prior evaluations focus primarily on clean benchmark datasets and do not assess common real-world challenges such as noise, missing values, and limited labelled data. We conduct a systematic evaluation across 17 binary classification datasets from UCI, OpenML, and Kaggle, covering domains including fraud detection, predictive maintenance, and healthcare. Six models are compared: HyperFast (default and tuned), Logistic Regression, LightGBM, XGBoost, and Random Forest. Experiments span 15 random seeds and 13 conditions per dataset, yielding 19,890 model evaluations. HyperFast (default) wins none of the 224 evaluated conditions and is significantly worse than the best-performing model in 206 cases. HyperFast-Tuned wins 30 conditions, primarily on highly imbalanced datasets under degraded data conditions, but requires between 81 and 16,191 times more inference time than Logistic Regression. Logistic Regression achieves 86 condition wins and LightGBM 61, together accounting for 66 Our findings indicate that training-free inference does not eliminate computational cost but shifts it from offline training to deployment-time inference. Across the evaluated datasets and robustness conditions, classical and gradient-boosted methods generally provide superior accuracy-runtime trade-offs, while HyperFast-Tuned offers advantages only in a limited set of challenging scenarios.