Automation of Mass Spectrometry Imaging Using Fisheye Cameras

Collecting mass spectrometry (MS) data of histological samples using a liquid micro junction-surface sampling probe is generally unrepeatable and time-consuming due to difficulties in manually positioning the probe as precisely as possible, and aligning the exact times of probe contact and MS data acquisition. Previous work has shown that real-world probing coordinates can be calculated from undistorted sample images captured by a fisheye camera. This project sought to develop an application that fully automates calculating real-world sampling coordinates from unwarped sample photos, and creating MS images using timestamped MS data. The application’s interface and backend image processing are implemented using Python’s PyQt5, OpenCV, and Numpy libraries. A fisheye camera and probe are attached to the robotic arm of a 3D printer and pointed towards the printer bed to capture images and collect MS data of samples. Fisheye camera photos must be processed through checkerboard detection to establish visual linearity. AprilTags can assist in determining the camera-to-probe millimetre offset using eye-in-hand calibration and 3D coordinate conversions. The chronological delay between probe movements and data collection can be found using an initial sampling point with a distinct MS profile. With these configurations, users can capture an unwarped sample photo, select a region of interest to collect data from, specify the sampling resolution, and initiate automated MS sampling using the specified parameters and calculated image-to-real-world probing coordinates. The application’s automated sampling and timestamping functionality successfully facilitates the generation of MS images of flat histological samples. The application calculates the physical camera-to-probe offset with sub-millimetre accuracy. Image-to-real-world coordinate calculation accuracy is dependent on the checkerboard pattern used to undistort the fisheye camera, environmental lighting, and AprilTag positioning for the camera-to-probe offset estimate. Future work will integrate conductive sampling for uneven sample surfaces and verify alignment of timestamps to MS data acquisition.