Holomedicine: Proof of the Concept of Interactive Three‐Dimensional Telemedicine

The need for telemedicine has been rapidly recognized, especially after the coronavirus disease 2019 (COVID-19) pandemic, which seriously affected individuals with Parkinson's disease (PD) by disrupting their regular doctor visits. Telemedicine may reduce unnecessary risks of infections,1 in addition to reducing costs and time to visit the hospital.2 However, the current telemedicine is based on two-dimensional (2D) technology, such as a video call system via laptops, desktop computers, tablets, or smartphones,3 which limits the observation of three-dimensional (3D) movements. To overcome the limitations of 2D telemedicine, we propose using the new concept of “holomedicine,” an interactive 3D hologram-based telemedicine, which consists of RGB-depth camera sensors (Kinect v2; Microsoft, Redmond, WA, USA) that capture a person's movements using infrared light and an augmented/mixed reality head mount display with a hologram monitor for checking the 3D-digitized person's data (HoloLens; Microsoft). The system consisted of two parts: the patient side and the physician side (Supplementary S1). The patient-side system captured the shape of a patient and sent that data as a 3D-digitized image to the physician side via a wireless network. The physician-side system received data from the patient-side system and reproduced the patient's 3D-digitalized shape in almost real-time using peer-to-peer networking, and vice versa. By using these systems in geographically distant locations, physicians could see patients in front of them as if they were sharing the same room because the augmented/mixed reality headset screens are transparent, in contrast with virtual reality headsets (Supporting Information Video SS1). In our preliminary study, we confirmed the feasibility to assess the motor score via the 3D telemedicine in a total 100 of patients with PD. There was a significant correlation between scores of the Movement Disorder Society–sponsored revision of the Unified PD Rating Scale part III obtained by a neurologist using that system and those measured in the face-to-face evaluation (r = 0.872). Moreover, an adequate intraclass correlation coefficient (2,1) of 0.646 was obtained for these scores (Supplementary S2). This mixed reality healthcare visit could lead to an evolutional change of the traditional hospital appointment because it approximates the in-person visit. The advantages of 3D over 2D technology are that the 3D technology produces a realistic atmosphere similar to a face-to-face visit, which may help establish additional trust between patients and doctors compared with 2D methods alone. Moreover, the 3D data can be recorded and automatically accumulated during a telemedicine visit, and used for further 3D motion analysis and for machine or deep learning to develop an artificial intelligence–based diagnostic assistance system.4 The biggest limitation of the current 3D telemedicine system is time and space resolution. Second, examinations that require physical contact, for example, rigidity and postural instability, are still impossible. Third, the head mount display may cause discomfort as a result of wear and potential virtual reality sickness. However, those limitations could be solved with technological innovations in device, network connections such as fifth-generation cellular network technology, combining with wearable devices, and future hologram technologies. Future research should include a comparison between 3D and 2D systems. Innovative research that improves the quality and accuracy of these devices should be accelerated. This study was supported by Grants-in Aid from the Research Committee of CNS Degenerative Diseases; Research on Policy Planning and Evaluation for Rare and Intractable Diseases; Health, Labour and Welfare Sciences Research Grants; the Ministry of Health, Labour and Welfare, Japan; and grants from Novartis Pharma (Novartis Research Grants; 2015, 2016, and 2019). The funders had no role in study design, data collection or analysis, decision to publish, or manuscript preparation. S.S. designed the study, analyzed the data, and wrote the initial draft of the manuscript. G.O. contributed to the funding, concept and design of the study, and data interpretation; assisted in the preparation of the manuscript; and critically reviewed the manuscript. S.C. developed the program and contributed to data collection. All other authors contributed to data collection and interpretation and critically reviewed the manuscript. All authors approved the final version of the manuscript and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.