Abstract Foundation models (FMs) promise to standardise predictive modeling across domains, yet their clinical value for tabular data remains unproven. To test this, we performed a large, fully reproducible benchmark of TabPFN, a leading FM for tabular prediction, against twelve established machine learning (ML) methods across twelve binary clinical tasks. Cohorts spanned 788 − 139,528 patients across diverse outcomes, including survival, metastasis, and disease status. Using standardized preprocessing, bootstrapping, and multiple performance metrics, TabPFN was generally competitive but did not consistently outperform strong ML baselines. It exceeded the best ML model in only 16.7% of tasks, with most area under the receiver operating characteristic (AUROC) differences within ± 0.02. TabPFN also incurred higher computational cost, with median runtimes 5.5× longer and practical reliance on GPU acceleration. These findings indicate that, for routine binary clinical prediction, TabPFN offers limited performance gains relative to optimized ML methods, while introducing significant efficiency trade-offs.
Shaktah et al. (Fri,) studied this question.