Long exposure convolutional memory network for accurate estimation of finger kinematics from surface electromyographic signals

Abstract Objective . Estimation of finger kinematics is an important function of an intuitive human–machine interface, such as gesture recognition. Here, we propose a novel deep learning method, named long exposure convolutional memory network (LE-ConvMN), and use it to proportionally estimate finger joint angles through surface electromyographic (sEMG) signals. Approach . We use a convolution structure to replace the neuron structure of traditional long short-term memory (LSTM) networks, and use the long exposure data structure which retains the spatial and temporal information of the electrodes as input. The Ninapro database, which contains continuous finger gestures and corresponding sEMG signals was used to verify the efficiency of the proposed deep learning method. The proposed method was compared with LSTM and Sparse Pseudo-input Gaussian Process (SPGP) on this database to predict the ten main joint angles on the hand based on sEMG. The correlation coefficient (CC) was evaluated using the three methods on eight healthy subjects, and all the methods adopted the root mean square (RMS) features. Main results. The experimental results showed that the average CC, root mean square error, normalized root mean square error of the proposed LE-ConvMN method (0.82 ± 0.03,11.54 ± 1.89,0.12 ± 0.013) was significantly higher than SPGP (0.65 ± 0.05, p < 0.001; 15.51 ± 2.82, p < 0.001; 0.16 ± 0.01, p < 0.001) and LSTM (0.64 ± 0.06, p < 0.001; 14.77 ± 3.21, p < 0.001; 0.15 ± 0.02, p = < 0.001). Furthermore, the proposed real-time-estimation method has a computation cost of only approximately 82 ms to output one state of ten joints (average value of 10 tests on TitanV GPU). Significance . The proposed LE-ConvMN method could efficiently estimate the continuous movement of fingers with sEMG, and its performance is significantly superior to two established deep learning methods.