Augmented reality visualization of biomechanical wall stresses on abdominal aortic aneurysms using artificial intelligence

Figures

• 

  Fig. 1.. A study presented by Phelps et al. 2017 surveyed patients while interaction with clinicians. The patients were surveyed an gave a score based on their understanding, accuracy, trust, anxiety, and overall satisfaction of their hospital visit when an image was not shown (verbal diagnosis given), 2D, and 3D medical images shown. It was found that patient understanding, accuracy, trust, anxiety, and satisfaction was statistically significant when comparing 2D image presented vs. no image, 3D image presented vs. no image (readapted from Phelps et al. 2017). Anxiety was not statistically significant when patients were shown the images of their diagnosis.

• 

  Fig. 2.. The aorta is a major conduit for oxygenated blood flow from the heart to the rest of the body. The abdominal aorta leads to the bifurcation to the lower limbs. When the terminal aorta begins to dilate 50 % or greater, it becomes defined as an abdominal aortic aneurysm (AAA). This condition is highly prevalent in about 2.5 million US citizens over the age of 65 with 200,000 new diagnoses each year. Clinicians will track the diameter through medical imaging surveillance before a maximum diameter threshold is met (5.0 cm for women, and 5.5 cm for men). After the threshold is breached, clinicians will provide clinical intervention through surgery. However, there are still a relatively large number of patients rupturing before this threshold, a critical event that leads to a death (75–90 % mortality). Biomechanics researchers have investigated the relationship between wall stress and wall strength to understand which patients may be at risk of rupture, a potentially better tool than simply measuring diameter.

• 

  Fig. 3.. A) Initial medical image for an AAA that can be images using computed tomography, magnetic resonance imaging, and ultrasound. The medical image data uses the DICOM header file embedded inside each individual 2D image file (that makes an entire 3D volume or stack in the axial direction). Image segmentation occurs here after the medical images are acquired. Automated segmentation (using a trained 2D U-NET) or manual/semi-automated image segmentation is used for this process before B) A 3D point cloud is generated from the manual/semi-automated or automated segmentation. The 3D point cloud uses pixel dimensions for X,Y, and Z directions. A scaling factor is applied using information from the DICOM header file (X and Y pixel spacing) and relative position (image patient position) for the distance between 2D axial slices. The scaling factors (pixels/mm) are applied to the 3D point cloud to convert into ‘real-world’ units. C) The 3D point cloud is converted into a 3D surface. This is achieved my computing the normal of each vertex or node and finding a neighborhood of points. A Poisson reconstruction is applied to create a 3D surface of the aneurysm wall. D) In a previous study,…

• 

  Fig. 4.. Stress prediction results using linear regression (traditional statistical method) and different machine learning models (ensemble boosted trees, decision trees, and neural networks. The figure shows the ground truth (from finite element analysis) on the first column, and the results of the various methods showing the anterior and posterior views. The wide neural network performed the best among the trained models. The wall stress map shows elevated stress banding on the posterior section of the aneurysm.

• 

  Fig. 5.. A) The rendered 3D models with the heat map baked onto the surface are exported and uploaded into the augmented reality system. There are three models loaded using the file system and operating system of the hardware (Microsoft HoloLens). The three models represent the stress predictions from a fine tree model, ground truth (stress analysis), and a trilinear neural network. B) A 2D image of the analytics used for the example 3D model with stress predictions (91st percentile for PWS, and 72nd percentile for MWS) can be visualized in the HoloLens environment while simultaneously viewing the 3D models. The data processed is from a database that is growing and can be used to compare a patient aneurysms to a growing population.