Patient-specific Cardio-respiratory Motion Prediction in X-ray Angiography using LSTM Networks

Abstract Objective. To develop a novel patient-specific cardio-respiratory motion prediction approach for X-ray angiography time series based on a simple long short-term memory (LSTM) model. Approach. The cardio-respiratory motion behavior in an X-ray image sequence was represented as a sequence of 2D affine transformation matrices, which provide the displacement information of contrasted moving objects (arteries and medical devices) in a sequence. The displacement information includes translation, rotation, shearing, and scaling in 2D. A many-to-many LSTM model was developed to predict 2D transformation parameters in matrix form for future frames based on previously generated images. The method was developed with 64 simulated phantom datasets (pediatric and adult patients) using a realistic cardio-respiratory motion simulator (XCAT) and was validated using 10 different patient X-ray angiography sequences. Main results. Using this method we achieved less than 1 mm prediction error for complex cardio-respiratory motion prediction. The following mean prediction error values were recorded over all the simulated sequences: 0.39 mm (for both motions), 0.33 mm (for only cardiac motion), and 0.47 mm (for only respiratory motion). The mean prediction error for the patient dataset was 0.58 mm. Significance. This study paves the road for a patient-specific cardio-respiratory motion prediction model, which might improve navigation guidance during cardiac interventions.