Human-Machine Collaborative Systems for Microsurgical Applications

Human-machine collaborative systems (HMCSs) are systems that amplify or assist human capabilities during the performance of tasks that require both human judgment and robotic precision. We examine the design and performance of HMCSs in the context of microsurgical procedures such as vitreo-retinal eye surgery. Three specific problems considered are: (1) development of systems tools for describing and implementing HMCSs, (2) segmentation of complex tasks into logical components given sensor traces of human task execution, and (3) measurement and evaluation of HMCS performance. These components can be integrated into a complete workstation with the ability to automatically “parse” traces of user activities into task models, which are loaded into an execution environment to provide the user with assistance using on-line recognition of task states. The major contributions of this work include an XML task graph modeling framework and execution engine, an algorithm for real-time segmentation of user actions using continuous hidden Markov models, and validation techniques for analyzing the performance of HMCSs.