Navigating foundation model selection in digital pathology through performance evaluation and tradeoff analysis

The rapid proliferation of digital pathology foundation models (FMs), spanning widely in architectural scale, pretraining data, and training strategy, poses a practical challenge for selecting the optimal model for a specific clinical application. To systematically evaluate these trade-offs, We benchmarked 6 FMs representing different model tiers and design profiles, including Lunit, Kaiko-Base, Phikon-v2, UNI2, Virchow2, and Kaiko-Midnight, across whole-slide image (WSI)-level classification, WSI-level survival prediction, and region-of-interest (ROI)-level classification tasks. Our results show that model performance is strongly task-specific, challenging the assumption that larger models are universally better. At the WSI level, Camelyon16 lymph node metastasis detection showed near-ceiling performance across models; Virchow2 achieved the highest observed AUROC and a statistically significant but small difference from the runner-up. For NSCLC subtype classification, Phikon-v2 achieved the highest observed AUROC, although its difference from the runner-up was not statistically significant. For NSCLC survival prediction, Lunit, the smallest model evaluated, achieved the highest observed five-fold cross-validated mean C-index and lowest mean rank, while Kaiko-Base provided the most stable prognostic performance; however, matched-fold analysis did not establish statistically significant superiority. ROI-level classification was evaluated using macro-averaged one-vs-rest AUROC. BACH and TCGA-UT showed near-ceiling performance among the leading models, whereas UNITOPATHO remained more discriminative and showed the clearest statistically supported separation, with Kaiko-Midnight significantly exceeding the runner-up Lunit. Kaiko-Midnight also achieved the highest observed AUROC on TCGA-UT, but its difference from UNI2 was not statistically significant. The Kaiko-Base versus Kaiko-Midnight comparison further showed that a larger model within a related development lineage can achieve higher observed ROI-level AUROC for selected tasks without necessarily improving WSI-level classification or survival prediction. Quantitative embedding-extraction benchmarking also revealed substantial deployment-relevant efficiency differences, with Kaiko-Midnight showing markedly lower throughput than Lunit under maximum feasible batch-size evaluation. Finally, training-data fraction ablations with 50 random seeds per fraction showed that model rankings can shift across data regimes and that low-data settings introduce substantial variability, particularly for WSI-level AMIL training. Together, these findings support a task- and data-regime-aware framework for selecting digital pathology FMs, balancing predictive performance, statistical uncertainty, stability, computational efficiency, and deployment cost.