Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation

Figures

• 

  Fig. 1. Outline of the application. Raw data acquired from the simulator is used to calculate relevant features, metrics of interest. Data obtained from participants who are at different stages of expertise is used to train a LSTM network. The trained algorithm provided continuous assessment, intelligent instructions, or risk warnings, depending on the output feature selected. Multiple algorithms are trained to demonstrate potential applications of the ICEMS.

• 

  Fig. 2. Flow diagram. AI: artificial intelligence. One trial data belonging to a neurosurgeon was not available.

• 

  Fig. 3. Simulated tumor resection tasks. Participants carried out two simulated tumor resection tasks, the simulated subpial tumor resection (a, b and c) 5 times and the simulated complex brain tumor operation (d, e and f) once, employing a simulated ultrasonic aspirator in the dominant hand and a simulated bipolar forceps in the non-dominant hand. Both instruments were activated by separate pedals. These tasks were designed with bleeding capacity to replicate the high-risk complex subpial brain tumor resection. (f) demonstrates cauterization using the bipolar forceps.

• 

  Fig. 4. Performance metrics. Sixteen performance metrics from five categories: safety, quality, efficiency, bimanual cognitive and movement, were extracted from the raw data. An LSTM network was trained inputting the 16-performance metrics, predicting expertise. The LSTM network was structured as regression model to predict expertise as a continuous variable from 1 (expert) to −1 (novice). N Newton, mm millimeter, t time (0.02 s).

• 

  Fig. 5. Average score of groups. When the performance of the participants was scored by the ICEMS, the average scores were: for experts (neurosurgeons, n = 14) 0.509; 95% CI [0.424–0.593], for seniors (n = 14) 0.258; 95% CI [0.114–0.402], for juniors (n = 10) −0.11; 95% CI [−0.358–0.139], and for novices (medical students, n = 12) −0.398; 95% CI [−0.545 −0.251]. Skilled and less skilled performance are represented in the y-axis by scores closer to ‘1’ and ‘−1’, respectively. Bars represent standard errors.

• 

  Fig. 6. Average score versus year of training in neurosurgery. The average score yielded a significant correlation with the trainees’ year of training (p = 0.005), increased by 0.092 per training-year, with a linear regression analysis. Blue dots represent the average score of each trainee, x axis represents year of training in neurosurgery. Resident participants’ neurosurgery training program was six years. Neurosurgical fellows were considered in 7th year in training.