Real-time 3D ultrasound in augmented reality accelerates training and narrows novice–expert performance gaps

Ultrasound imaging requires users to infer three-dimensional anatomy from two-dimensional slices, imposing steep training demands that limit broader adoption. Here we present AR-VIU, a mixed-reality platform that streams real-time volumetric ultrasound as point-cloud renderings into an augmented-reality headset with true-scale spatial registration. To isolate the contributions of volumetric imaging and immersive display, we tested four conditions—two-dimensional imaging on a screen, two-dimensional imaging in augmented reality, three-dimensional imaging on a screen, and three-dimensional imaging in augmented reality—in a controlled study with 18 participants (9 novices, 9 experts). Participants performed object recognition and localization tasks. The augmented-reality volumetric system was associated with the highest accuracy, lowest variability, and near-elimination of the novice-expert performance gap. These results demonstrate technical feasibility for real-time three-dimensional ultrasound in mixed reality and establish an evaluation framework for perceptual and cognitive performance in clinically relevant scenarios, with near-term applications in training and education. Jason Hou and colleagues present AR-VIU, a mixed-reality platform that streams live 3D ultrasound into an augmented-reality headset. In a controlled study, the system improved recognition and localization accuracy while narrowing the gap between novice and expert users.