Model-free Cardiorespiratory Motion Prediction from X-ray Angiography Sequence with LSTM Network

We present a novel model-free approach for cardiorespiratory motion prediction from X-ray angiography time series based on Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN). Cardiorespiratory motion prediction is defined as a problem of estimating the future displacement of the coronary vessels in the next image frame in an X-ray angiography sequence. The displacement of the vessels is represented as a sequence of 2D affine transformation matrices allowing 2D X-ray registrations in a sequence. The new displacement parameters from a sequence of transformation matrices are predicted using an LSTM model. LSTM is a particular form of Recurrent Neural Network (RNN) architecture suitable for learning sequential data and predicting time series. The method was developed and validated by simulated data using a realistic cardiorespiratory motion simulator (XCAT). The results show that this method converges quickly and can predict the complex motion in the angiography sequences with irregularities. The mean values of prediction error over all the patients are approximately 0.29 mm (2 pixels) difference for the combination of both motions, 0.51 mm (3.5 pixels) difference for cardiac motion and 0.44 mm (3 pixels) difference for respiratory motion.