Opportunities and obstacles in harnessing intraoperative data

Robotic surgery utilization has increased rapidly over the past decade, a trend that is well recognized across surgical specialties, with general surgery representing the largest and fastest-growing contributor. 1 This expansion has been reflected by the transition of several complex general surgical procedures-such as pancreatectomy, colorectal surgery, and esophagectomy-from laparoscopic to predominantly robotic approaches². Similarly, minimally invasive hernia repair has seen substantial growth, with robotic inguinal and ventral hernia repairs increasing more than 40-fold between 2012 and 2018 11,2 .A consequence of the rapid expansion of robotic surgery is the generation and capture of novel, high-dimensional data. The introduction of robotic platforms in the OR offers the potential to generate and capture rich intraoperative data streams, including modalities rarely or never captured previously, such as video, haptics, kinematics, audio, robotic usage metadata, and more. With the advent of this data, the previous "black box" of the intraoperative period has the potential to become more measurable and analyzable. High-fidelity and time-stamped data with synchronized audiovisual information collected over time can create large datasets that can ultimately be linked to clinical outcomes and identify best practices.The availability of this data provides new opportunities for improvements in patient care, education, quality improvement, innovations in intraoperative AI, and more. Current research on intraoperative AI applications is in early developmental stages, demonstrating how AI applications can enhance surgical precision through motion analysis as well as provide decision support through surgical phase recognition and anatomical landmark identification 34 , but this intraoperative data will surely prove to be instrumental in advancing the capabilities of these tools. Most existing intraoperative AI systems have the capability to provide simple assistance, with very few (3%) providing conditional decision-making 4,5 . However, as higher-quality, task-specific intraoperative data are collected and analyzed, these data may help catalyze advancement of efforts to enable robots to perform select autonomous surgical tasks 6 .3) Data for Quality Improvement Data generated and collected on robotic platforms may also aid in quality improvement efforts in the form of retrospective analyses to ensure best practices are being met. For example, intraoperative data has already been used to ensure the critical view of safety is being achieved during a cholecystectomy or using ambient operating room sensors to collect audiovisual data to ensure a surgical timeout is always being performed 7,8 . Metadata generated by how physicians interact with surgical technology contains valuable signals for understanding and improving healthcare delivery. For example, studying audit log data from how physicians use electronic health records has provided great insight into understanding how patient care is delivered 910 . Thus, studying metadata from how users are interacting with the robot could give tremendous insight into how to improve intraoperative processes.On the side of clinical research, the data generation of robotic surgery has created a rapid pace of evidence development. This has encouraged increased research in the form of retrospective studies and database reviews 11 . The abundance of automated data generation unique to robots creates large and easily accessible datasets that allow researchers to analyze observational data. However, there is a need for well-designed prospective and randomized studies to create higher-quality evidence of the impact of this data.In the context of education, data leveraged from robotic surgery can play a role in structuring surgical development, assessing skills, and monitoring performance improvement. Automated metrics derived from robotic tools can provide objective data on technical parameters, motion, and time to help trainees improve 12 . Data-driven insights from robotic surgeries can offer opportunities for standardized, objective skills assessment and feedback, supporting surgical education. 1314 .These are among some of the many opportunities robotic surgery has to offer. There remain obstacles that prevent its true potential from being unlocked. The potential of this data and the involvement of multiple stakeholders in generating this data raise the issue of how this data should be governed so that it can be used for all of the potential benefits listed above. While not comprehensive, some important elements of establishing a governance infrastructure for this data are standardization and interoperability. Presently, there is a lack of a standardized and universally adopted ontology for intraoperative data in robotic surgery. Several groups, including the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES, as well as industry groups such as Intuitive Surgical, are working to advance the ontology and annotation of intraoperative events through the development of standardized frameworks for surgical phases, tasks, actions, and gestures 15 . Standardized frameworks that exist outside of surgery can serve as models. A standard used in radiology called Digital Imaging and Communications in Medicine (DICOM) aims to ensure interoperability between different manufacturers' devices and set guidelines for managing, storing, and transmitting medical image data 16 . Data standards adapted to intraoperative data could similarly be beneficial.Additionally, data interoperability -the seamless transfer and aggregation of data across vendors, researchers, and healthcare systems -will be an important feature of this data to ensure it can be used productively. Interoperability is complementary to standardization: while standardization ensures data is in a comparable format across various vendors and institutions, interoperability ensures that there are resources to allow the data to be aggregated into larger datasets and used for more powerful analyses. As an example, the development of interoperability in the setting of EHR data may offer lessons for what this might look like for intraoperative data. There is a well-developed regulatory environment around the sharing of electronic health information (EHI), which is data commonly found in EHRs.The 21st Century Cures Act established rules around "information blocking" that prevent healthcare providers, health information technology developers, exchanges, and networks from interfering with authorized access to EHI 17 . The Office of the National Coordinator for Health Information Technology (ONC) Final Rule implemented these provisions and specified that there must be standardized application programming interfaces (APIs) that enable data sharing 18 . Finally, interoperability standards, including Health Level Seven (HL7) and Fast Healthcare Interoperability Resource (FHIR) were established to facilitate the integration, exchange, sharing, and retrieval of EHI 19 .The rapid expansion of robotic surgery means that there has been an influx of new digital technology into the operating room that is capturing and generating surgical data that has never been done at this scale. There is great potential in using this data as it can be crucial to surgical skill improvement, patient care, and research efforts to improve surgical outcomes. A rate-limiting step to harnessing this data is to proactively ensure it is standardized and shareable. This is the first step to ensure a collaborative and cohesive environment that promotes innovation and inspires researchers and surgeons to optimize surgical care.