What question did this study set out to answer?

The research aims to analyze the uncertainty in machine learning models used for MRI data analysis and propose methods to improve their reliability.

June 12, 2026Open Access

The Sim-to-Real Uncertainty Gap in Quantitative MRI: Characterization, Benchmark, and Counterfactual Correction

Key Points

The research aims to analyze the uncertainty in machine learning models used for MRI data analysis and propose methods to improve their reliability.
Developed a benchmark called qMR-FailureBench with 60,000 simulated MRI signals and five evaluation tasks.
Applied a calibration step using 5% of real scan data to improve model output.
Assessed model performance on real patient data without fine-tuning.
Calibration reduced measurement errors by 39.6%.
Models failed to distinguish between reliable and unreliable predictions without calibration.
Confidence estimates for predictions significantly improved post-calibration.

Abstract

Machine learning models trained on simulated MRI data are increasingly used to extractquantitative measurements from real brain scans. But can these models tell us when they mightbe wrong? We provide evidence that they usually cannot. When tested on real patient datawithout any fine-tuning, the models still produce reasonable measurements, but their confidenceestimates break down—they can no longer distinguish reliable predictions from unreliable ones.We call this the sim-to-real uncertainty gap. We demonstrate that this gap can be fixed witha quick calibration step using only 5% of real scan data. To help the community study and solvethis problem, we release qMR-FailureBench, a standardized benchmark of 60,000 simulatedMRI signals with five evaluation tasks. We also show that our system can not only detectunreliable measurements, but identify why they failed and attempt to correct them, reducingerrors by 39.6%

Read Full Paperexternally

Demander à l'IA

Bookmark

View Full Paper