Multiview deep learning improves detection of major cardiac conditions from echocardiography

Abstract Medical imaging often captures multiple two-dimensional views of three-dimensional anatomic structures, but most artificial intelligence (AI) models analyze two-dimensional data. Here we show that integrating multiple imaging views using a single AI model can improve diagnostic performance. We developed a deep neural network (DNN) architecture that combines information from multiple video views simultaneously. Using echocardiogram data from the University of California, San Francisco, and the Montreal Heart Institute, we applied our multiview DNN approach for three primary demonstration tasks: detecting any left or right ventricular abnormality, diastolic dysfunction, and substantial valvular regurgitation. Across various tasks, our multiview DNNs improved discrimination as measured by the area under the receiver operating characteristic curve by 0.06–0.09 compared to DNNs trained on any single view. This demonstrates that AI models that can combine information from multiple imaging views simultaneously can better capture complex anatomy and physiology for certain tasks, underscoring the value of a multiview paradigm for AI in medical imaging.