Recurrent Neural Network Methods for Extracting Dynamic Balance Variables during Gait from a Single Inertial Measurement Unit

Figures

• 

  Figure 1. The marker set in (A) anterior and (B) posterior view. The marker positions are anterior superior iliac spines (RASI and LASI), posterior superior iliac spines (RPSI and LPSI), greater trochanters (RTRO and LTRO), mid-thighs (RTHI and LTHI), medial and lateral epicondyles (RMFC, RLFC, LMFC and LLFC), heads of fibulae (RSHA and LSHA), tibial tuberosities (RTT and LTT), medial and lateral malleoli (RMMA, RLMA, LMMA and LLMA), navicular tuberosities (RFOO and LFOO), fifth metatarsal bases (RTOE and LTOE), big toes (RBTO and LBTO) and heels (RHEE and LHEE), and condylar processes of the mandibles (RHead and LHead), acromion processes (RSAP and LSAP), the seventh cervical vertebra (C7), medial and lateral humeral epicondyles (RUM, RRM, LUM and LRM), and ulnar styloids (RUS and LUS) [62,63].

• 

  Figure 2. (A) Experimental photo showing a typical subject with a waist-worn IMU stepping on force plates during level walking. The IMU with an embedded coordinate system is also shown in the inlet. The COM–COP vector forms the inclination angles (IA) with the vertical: (B) sagittal IA (α) and (C) frontal IA (β). Mean curves of the IA and their rates of change (RCIA) are also shown. HS: heel-strike; TO: toe-off; CHS: contralateral heel-strike; CTO: contralateral toe-off.

• 

  Figure 3. Flowchart of extracting dynamic balance variables from data of a single inertial measurement unit (IMU) with four recurrent neural network models, namely uni-LSTM, bi-LSTM, uni-GRU, and bi-GRU (yellow box). The input data for the models were three-dimensional angular velocities and linear accelerations recorded from the IMU (blue box). The desired outputs of the models were balance variables, namely the IAs and RCIAs in both sagittal and frontal planes (green box). The sensor data and balance variables were normalized to the gait cycle. Each model utilized the normalized IMU data as input and made accurate predictions for the desired IAs and subsequently calculated RCIAs by differentiation of IAs once.

• 

  Figure 4. Internal structures of two recurrent neural network (RNN) cells, namely the (A) long short-term memory (LSTM) network and the (B) gated recurrent unit (GRU) [73,76]. LSTM employs a forget gate (red box) to selectively eliminate irrelevant information from the current inputs (xt) and previous hidden state (ht−1). An input gate (blue box) is utilized to update the previous cell state (ct−1) to the current cell state (ct), while an output gate (green box) generates the current hidden state (ht) and outputs (yt). GRU simplifies LSTM by reducing the number of gates. GRU integrates a reset gate (purple box) to discard irrelevant information from the previous hidden state (ht−1), yielding a modified hidden state. An update gate (yellow box) is used to combine the modified hidden state with the hidden state (ht−1) and current inputs (xt) into the current hidden state (ht) and outputs (yt). These structural designs enable RNNs to capture and handle long-term dependencies in time series analysis effectively.

• 

  Figure 5. The internal structures of recurrent neural network (RNN) layers, namely the (A) uni-directional and (B) bi-directional layers [77]. The uni-directional layer processes input sequences sequentially, updating hidden states based on previous states. The bi-directional layer enhances this by processing sequences in both directions, combining forward and backwards hidden states. Uni-directional layers capture past information, while bi-directional layers capture dependencies from both past and future contexts.

• 

  Figure 6. Effects of loss functions (standard MSE vs. weighted MSE), cell types (LSTM vs. GRU), and flow of information (uni-direction vs. bi-Direction) on the prediction errors (RMSE) of the sagittal and frontal inclination angles (IAs) (A,B) and rates of changes of IAs (RCIAs) (C,D) for the four machine learning models (i.e., uni-LSTM, uni-GRU, bi-LSTM and bi-GRU). Error bars are standard deviations. PL: p-values for loss function factor (i.e., uni-LSTM, uni-GRU, bi-LSTM and bi-GRU); PC: p-values for cell type factor. p-values for the direction factor are all greater than 0.05.

• 

  Figure 7. Effects of loss functions (standard MSE vs. weighted MSE), cell types (LSTM vs. GRU), and flow of information (uni-direction vs. bi-Direction) on the rRMSE (relative RMSE) of the sagittal and frontal inclination angles (IAs) (A,B) and rates of changes of IAs (RCIAs) (C,D) for the four machine learning models (i.e., uni-LSTM, uni-GRU, bi-LSTM and bi-GRU). Error bars are standard deviations. PL: p-values for loss function factor (i.e., uni-LSTM, uni-GRU, bi-LSTM and bi-GRU); PC: p-values for cell type factor. p-values for the direction factor are all greater than 0.05.