An adaptable robotic arm-based research platform for the capture of multi-view datasets

Accurate and reliable depth reconstruction within surgical scenes is essential to downstream computer- assisted tasks ranging from pre-operative planning to augmented reality (AR) surgical guidance and simultaneous Localization and Mapping (SLAM) 1. However, in surgical settings, several challenges hinder the acquisition of multi-view datasets essential for depth reconstruction systems. These include, in addition to ethical issues, several domain-specific technical challenges such as dynamic surfaces, the presence of occlusions such as blood, smoke, and surgical instruments, and the presence of specular reflections. As a result, the vast majority of research considering depth information in a surgical context avoids multi-view data capture completely, mostly due to its inability to be used in minimally invasive surgery (MIS). Instead, many existing approaches for capturing depth data in a surgical setting often involve complex acquisition techniques, such as utilizing a CT scanner for obtaining ground truth depth for comparison 2. This severely limits the scalability of the method, as well as the volume of data that can be captured. Other approaches have used structured light systems 3 or structure from motion techniques 1, 4 for depth reference, though the data captured through such systems sometimes exhibit unacceptable levels of noise. When developing optical systems or algorithms for depth imaging in both open-field surgery (OFS) and MIS, it is difficult to assess and evaluate the effectiveness of those systems without solving the logistical and ethical issues of using the system on animals or humans. It would be beneficial to have a system that can have any camera attached to it for the ability to quickly create datasets for testing.Therefore, in this work, we present a simple, cost- effective, and camera-agnostic robotic arm system for the acquisition of multi-view data. The system can be used to capture large quantities of data for developing surgical depth estimation systems and, with further development, may have the potential to be used in real-time settings.