Why we need to maintain a critical view on big data and artificial intelligence predictions

Artificial intelligence (AI) and machine learning are widely promoted as transformative tools for medical practice, yet their impact in daily rheumatology remains limited. This review examines the gap between expectations and reality using historical parallels, conceptual considerations, and recent methodological evidence. Experiences with antioxidant supplementation, vitamin D, the microbiome, and the Human Genome Project illustrate a recurring pattern: early studies report large effects that diminish or disappear in larger, higher-quality studies. Meta-epidemiological work and the 'cursed auction' analogy explain why early and small studies systematically overestimate effects. Conceptually, individualized clinical risk remains a group-based construct, constrained by the reference class problem and irreducible uncertainty. Methodologically, many AI models in rheumatology suffer from small and heterogeneous datasets, overfitting, inadequate handling of missing data, poor calibration, and limited external or prospective validation. The failure of COVID-19 prediction models and the neutral trial of the Ada diagnostic assistant in rheumatology illustrate how strong retrospective performance often collapses in real-world use. In contrast, AI performs well in high signal-to-noise domains with abundant, structured data. Overall, AI can generate valuable insights and support narrowly defined tasks, but it cannot yet overcome the fundamental limits of noisy clinical data and group-based risk. Progress in rheumatology will require realistic expectations, large representative datasets, transparent methods, rigorous validation, and a focus on robust, interpretable tools that improve decisions for populations and well-defined patient subgroups rather than precise individual prediction.