Enhancing Automated Mass Spectrometry Sampling with LMJ-SSP and Fisheye Camera-Based Computer Vision

Earlier work on this project developed a software system that utilized computer vision to guide automated mass spectrometry sampling with a liquid micro-junction surface sampling probe (LMJ-SSP) mounted on a retrofitted Prusa 3D printer. However, this system struggled with accurate probe localization in 3D space, required continuous manual intervention, and was highly sensitive to small variations in image conditions. These limitations prompted the need for a more precise, robust, and fully automated sampling workflow. This research focused on refining that system, including its software, fisheye camera, and lighting setup. For new transformations, the software now requires only one checkerboard image to compute the camera’s intrinsic and extrinsic parameters and correct fisheye-induced barrel distortion. Four images of an AprilTag’s corners are then used to localize the camera, probe, base, and sample within a 3D coordinate frame. Once calibrated, users can define a region of interest (ROI) and set spatial sampling parameters such as sampling frequency. For repeated use, saved transformation profiles can be loaded and applied directly to live camera feeds. This streamlined process achieves sub-millimeter probe localization accuracy (<0.5 mm) with minimal user input. To mitigate issues with lighting variability in captured images, a standardized blue background was introduced, improving checkerboard visibility and unwarping consistency. A polling-based communication architecture was also implemented to synchronize probe actions with data collection, eliminating problems associated with temporal latency and improving the alignment between probe positions and chemical signal data. The user interface was further redesigned for flexibility across screen sizes. The updated system is a more accurate, reproducible, and user-friendly platform for histological sampling. Future work will focus on integrating conductance sampling methods and introduce a non-linear region-of-interest (ROI) selection tool.