Virtual reality spine surgical training in Singapore: a preliminary study

INTRODUCTION Spine surgery is a field of practice in medicine that is technically demanding, as it requires high precision and efficiency. Therefore, training in this specialty has to be rigorous to ensure patient safety and good clinical outcomes. However, there is no formal spine residency programme in many parts of the world, including Singapore, where spinal surgery is considered part of the orthopaedic surgery or neurosurgery residency. This has led to inadequate exposure to spinal surgery among residents, with large-scale resident surveys showing a lack of confidence in incorporating spinal surgery in clinical practice.[1] Furthermore, surgeons are often reluctant to give residents the opportunity for hands-on practice due to concerns of increased complications and operative duration. While it may be partly justifiable due to concerns of patient safety,[2] this further compromises the ability of our institutions to nurture competent spine surgeons. Virtual reality (VR) technology is increasingly being explored as an innovative and effective way to supplement and support training in various fields of surgery,[3,4] as it is able to provide vivid three-dimensional models for teaching anatomy,[5] as well as realistic first-person surgical simulations. By replicating the immersive operating theatre set-up and step-by-step procedures, VR technology better engages the learner and effectively expounds many aspects of surgical training.[6] In addition, the reproducibility of VR simulations allows the creation of a standardised curriculum that includes rare complications and uncommon surgical procedures, which residents may not normally be exposed to, further increasing the potential of VR to add value to surgical education.[7] Our team developed a VR spinal surgery training simulator to supplement residents in their spine surgical training via viewing of immersive surgery recordings in a VR headset. Quiz questions were incorporated in the VR session to prompt students to review learning content, highlight key knowledge and test understanding of what has been taught.[8] The questions could either be posed to the residents after the entire surgery has been replayed ('out of game') or during the surgery replay in a more interactive way to reinforce learning ('in game'). Through this, we aimed to show that VR is able to provide a more immersive learning experience for residents and is at least as effective as two-dimensional (2D) videos in teaching concepts. The 2D videos were chosen as the basis of comparison, as they are a widely used and well-validated resource in medical education.[9,10] By having both in-game and out-of-game quizzes, this study also aimed to examine whether there are any differences in effectiveness of quiz placement. METHODS This project involved programming with Unity software (Unity Technologies, San Francisco, CA, USA) to develop an orthopaedic training system and the adoption of four Oculus Quest VR headsets (Facebook Technologies, LLC, Menlo Park, CA, USA) to replay the spine surgery recordings within the system, followed by a preliminary user study to test the feasibility of using VR in the teaching of spine surgery. The project was executed by five engineering undergraduate students under the supervision of a senior lecturer from the Department of Engineering at National University of Singapore. All residents from the Department of Orthopaedic Surgery at the National University Health System (NUHS) were recruited to participate in the user study. Subjective feedback was also obtained from the programme director of the NUHS residency programme and its senior faculty members. User study types Users were randomised using a lot drawing method to test four different teaching prototypes (A–D) to investigate two independent variables: the mode of learning (VR vs. 2D display) and the method of assessment (in-game vs. out-game assessment), and to obtain feedback on each form of teaching method. The four prototypes are as follows: (1) User study prototype A: VR headset video with an in-game assessment; (2) User study prototype B: VR headset video with an out-game assessment; (3) User study prototype C: 2D screen video with an in-game assessment; (4) User study prototype D: 2D screen video with an out-game assessment. The users watched a recording of a posterior lumbar laminotomy with discectomy on either an Oculus Quest VR headset or a 2D screen of a laptop. Then, they would either take an in-game quiz assessment, where a total of 13 multiple-choice questions (MCQs) would appear real time at relevant points in the video, or an out-game assessment, where MCQs would consecutively appear at the end of the video. For consistency, each user was allowed one playthrough of the recorded surgical procedures. Surgery recording and video playback We recorded the surgery on a lumbar spine model (Synbone AG, Malans, Switzerland) perfomed by a spine surgeon (first author, HHWD), using an Insta360 Evo camera (Insta360 Arashi Vision Inc., Shenzhen, China) that was mounted stationary on a tripod. To achieve a recorded video that best emulates the first surgeon's perspective, the camera was placed close to the eyes of the surgeon at the face level.[11] This is especially important for spine surgeries where the incision can be small with a long and narrow surgical corridor that precludes reliable video capture of the surgical field of view. This position of the camera, with its sensor clamped perpendicular to the surgeon's line of sight, also resulted in the smallest lens distortion recording. During the surgical filming, a microphone (Rode Video Mic Pro+; Rode Microphones, Sydney, Australia) was placed facing and close to the surgeon's mouth to enable clear recording of verbal commentaries, thus minimising unwanted noise. The recorded video was processed into a VR180 format that could be played back in the VR headset using a 180° stereoscopic view, which was chosen to limit the viewer's attention to our desired range.[12] This was to allow residents to focus on key areas in their learning for better outcomes. The recorded surgery was also divided into key steps, which allowed users to easily access and begin from specific parts [Figure 1].Figure 1: Snapshot of the virtual reality interface shows key surgical steps that users can choose to view.Virtual reality user interface and quiz To promote further interaction with the VR device, an optional screen window that portrays a 2D magnified view of the surgery was available. Modal windows in the form of quiz questions and non-modal popovers providing information on surgical instrument-in-use annotations were made possible when watching the surgery video.[13] An additional screen with in-depth descriptions and highlights of the surgical procedure, as well as an interactive anatomical spine with popovers to highlight spinal parts being operated on were also included. Residents who were allocated to the in-game group would encounter quiz questions throughout the relevant time points. These questions could be completed as they appeared at the end of each section. The MCQs were validated by the spine surgeon with the aim of testing the user's understanding of the surgery procedure. The method of scoring for the quiz takes into consideration recommendations from the HackerRank support centre website,[14] such that the marks allocated to each question were divided among the number of correct answers within the question. For example, if a 1-mark question involved two correct answers and the user chose two answers of which only one was correct, the user was awarded 0.5 marks. If more than two answers were chosen, no marks were awarded. A 1-min long briefing video was shown to allow users some time to familiarise themselves with the VR system. This interface served to remove confounding factors that might result from a new user being unfamiliar with how to operate the VR system. Residents were allowed to ask questions before using the designed prototype. Questionnaires Before the standardised briefing and use of the VR prototype, each user completed a prestudy self-administered questionnaire consisting of eight questions designed by the authors (YYE and FZNA). The questionnaire collected information on users' (a) demographics, (b) past experiences with VR, if any, (c) the year of their residency training, and (d) their health condition, in particular previous episodes of motion sickness.[15] A poststudy self-administered questionnaire consisting of 18 questions, (also designed by YYE and FZNA), was completed at the end of the study. This was to evaluate VR learning in terms of (a) video clarity, (b) user experience and (c) user comfort.[16] Each item was assessed on a Likert scale from 0 to 4, with 0 being 'Very Bad' and 4 being 'Excellent'. Thereafter, qualitative feedback was obtained to gain further insights regarding the project with three specific questions: "Was there any difference between watching a 2D video and using a VR headset?", "Do you think this VR system has potential to be useful in training, and if so, how?" and "Are there any further improvements you would recommend for this VR prototype?". The completed questionnaires were collected via interview by any two of the five undergraduate engineering students involved in the project. Statistical analysis Questionnaire and quiz data were compiled using Microsoft Excel for Office 365 MSO 32-bit version (Microsoft Corp., Redmond, WA, USA). Descriptive statistics of categorical and continuous variables were represented as numbers with percentages and means with standard deviations, respectively. Statistical analysis was done using IBM SPSS Statistics version 23.0 (IBM Corp, Armonk, NY, USA). A P value < 0.05 was considered as statistically significant. A two-sided unpaired t- test was performed to compare the learning environment (VR vs. 2D) and the type of assessment (in game or out game). Any available qualitative feedback was also recorded and shown. RESULTS All orthopaedic residents (n = 26, 84.6% males and 15.4% females) within the NUHS cluster participated in this VR project, took the quizzes and completed the prestudy and poststudy questionnaires [Table 1]. Their mean age was 31 ± 2.35 years. Majority (84.6%) of the users had not experienced motion sickness in the past. Although 53.8% had prior experience with VR through games or surgical simulations, none were frequent VR users and none had participated in VR-related activities over the 3-month study period.Table 1: Comparison of poststudy questionnaire results between VR and 2D users.In terms of quiz scores, users watching the video from a 2D display scored significantly higher when taking an in-game assessment compared to an out-game assessment (P = 0.039). However, when users were using the VR headset, the in-game and out-game assessment quiz scores did not show any significant difference (P = 0.907). Interestingly, while no difference in quiz scores was found between the VR and 2D display users for out-game assessments, VR users had lower scores compared to 2D display users for in-game assessments (P = 0.026). The poststudy questionnaire showed that VR users felt a higher 'sense of presence' (i.e. immersion in the surgical scenario) and had more 'engagement' watching the surgical recording as compared to 2D display users (P = 0.001 and 0.014, respectively) [Table 1]. However, there were no significant differences in perceived 'visual clarity' (P = 0.122), 'textual clarity' (P = 0.530), learning difficulty (P = 0.386) and usage difficulty (P = 0.290) between the scores provided by VR and 2D display users. Selected feedback from participants on the VR experience is shown in Table 2.Table 2: Qualitative feedback from participants (N=26).DISCUSSION This is the first study that incorporates VR technology into spine surgical training for residents. It includes an objective assessment of knowledge in the form of a quiz, and a user questionnaire that collected quantitative and qualitative feedback from the residents. With the rapidly evolving technological landscape and learning environment, this study represents an important first step. While the elements of assessment in this innovative approach are brief, this initiative sets the stage for integrating technology and surgical training, which is a course that must be pursued given the rapidly evolving learning environment and cultures of our new generation. Particularly in the field of spine surgery, where mistakes are poorly tolerated and technical skillsets are demanding,[17] the merits of VR technology make it an attractive and promising teaching method. These advantages include realistic, first-person perspective learning,[18] the ability to make mistakes without the fear of causing catastrophic complications, infinite repetitions for practices, and opportunities to witness rare conditions and procedures. In our study, the possibility of integrating VR into spinal surgery training was not an issue as users understood the recorded videos and scored equally in their postgame quizzes for both the traditional 2D and VR methods. Studies have proven that the learnability and usability of different applications and platforms vary.[19] Thus, for the purpose of the study, the number of clicks and steps required to play the video for both VR and 2D display methods were kept to the minimum and equal for both. The only difference was the use of a VR hand controller instead of the traditional mouse to navigate and click. Higher quiz scores for in-game assessment using 2D display method compared to out-game assessment scores suggest that in-game assessment is more likely to aid learning through delivering information in a timely manner. However, this could also be a result of the shorter time required for the user to retain the information obtained from the video and may not reflect long-term knowledge retention. The ability of VR to enhance long-term knowledge retention, as well as information assimilation and application, can thus be an important topic to explore in future studies. Based on the poststudy questionnaire responses, it is clear that residents felt that VR learning provides the same clarity as 2D display in learning (both visually and textually), without making the process more difficult. Visual clarity is of particular significance, as recognition of anatomical structures during a surgical procedure is key for performing a safe and successful surgery. In addition, the objective of creating a close-to-real-life simulated learning from first-person perspective appears to be possible, with residents feeling more immersed in the VR learning environment. This is reflected by the higher scores obtained from VR users in areas of 'sense of presence' and 'engagement'. The engaging VR environment could also explain why VR users may have performed more poorly in their in-game quiz scores than 2D display users, as VR users may be more fixated on the surgical simulation and pay less attention to other information that may not directly impact the simulation. However, this is not necessarily a weakness, as the purpose of incorporating VR is to create an immersive environment to facilitate learning of technical skills and could just reflect that VR teaching will still need to be utilised together with conventional teaching methods to boost theoretical knowledge. On the other hand, the lower quiz scores from VR users could also be due to insufficient time given to residents to adjust and acclimatise to the new VR environment as per the qualitative feedback from participant 16 [Table 2]. This was also a common comment during the interview session. However, we anticipate that with increased integration of technology in the field of medicine, clinicians will become increasingly familiar with VR, such that acclimatisation will become less of a concern. In general, the subjective feedback received during the interview session was positive, with many residents eager to see future integration of VR into their medical training. They also expressed the benefits of having more immersive training for better visualisation and learning purposes. Two residents expressed their personal preference for the current methods of training, as they felt that VR did not add value to their learning, and that the VR headset was too heavy. However, even among residents who personally preferred conventional learning, there was still a consensus that the ability to review surgeries in VR is helpful. Many also provided valuable insights on areas of improvement, for example, improving the environment by conducting the surgery in a busy operating theatre. There are several limitations to this study. First, the study had a relatively small sample size of 26 residents, which increases susceptibility to sampling biases and limits its applicability. In Singapore, residency is offered in three healthcare clusters (NUHS, SingHealth, National Healthcare Group) and the number of residents in each cluster is regulated by Ministry of Health Holdings, with each cluster hosting approximately one-third of all orthopaedic trainees. As such, by capturing all residents in the NUHS cluster, our sample size sufficiently represents residents in Singapore and can serve as the basis of a preliminary study before more users are recruited for larger studies. Secondly, we acknowledge that the quiz used may not truly assess the extent of knowledge of the resident. As the study design was kept simple to facilitate a feasibility trial, the quiz was relatively brief (13 questions), testing only a limited scope of knowledge. Furthermore the strength of VR lies in its ability to impart procedural skills, and thus, a quiz may not be the best modality to assess whether the residents' procedural skills improved after the session. More comprehensive testing is thus necessary to fairly assess the strengths and weaknesses of VR as an educational tool. Lastly, the quiz did not account for varying study tactics, levels of prior knowledge and personal experiences with 2D videos/VR among the residents, which may impact the quiz performance and feedback. Nonetheless, the small P- values (0.026) obtained when comparing the difference in scores between VR and 2D users during in-game assessments suggested that there is still a significant difference. To address problems with varying levels of prior knowledge, future studies could also include pre- and post-VR assessments and analyse the change in quiz scores to determine if the assessment has effectively imparted new knowledge to the participants. We also believe that a consensus obtained among residents with regards to VR being a valuable add-on tool is encouraging and reflects its potential to benefit a wide variety of learners. Beyond addressing limitations in further studies, we also hope to explore other strengths of the modality that have not been explored in this study. These include standardised medical training via the development and provision of reproducible learning content and exposing residents to multiple kinds of rare complications or surgical procedures, which they are often not exposed to due to safety and technical constraints within the spine posting. In the future, our interface may also be further improved to increase user-friendliness and the fidelity of our simulation. In conclusion, while VR training is still in its infancy, with the advancement of technology and the evolution of learning cultures, it displays great potential in improving current teaching methods.[20] Its numerous advantages in providing safe, repeatable and reproducible simulations of realistic surgical simulations are of particular appeal in the field of spine surgery. Our preliminary study shows that the implementation of VR in spine training is feasible and advantageous, though additional material may be required to supplement participant's learning. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest.