Deep Learning-Based Optimal Smart Shoes Sensor Selection for Energy Expenditure and Heart Rate Estimation

Figures

• 

  Figure 7. Structure of the proposed model. The shoe data from 20 channel sensors are fed into the input of the model and the channel-wise attention layer increases the intensity of the significant channels. The spatial features from the multi-channel data are extracted using DenseNet, and the temporal features are produced through Bi-GRU. Finally, HR and EE are estimated after the global average pooling (GAP) layer.

• 

  Figure 1. Overview of the system architecture for EE and HR estimation.

• 

  Figure 2. Locations of the sensors in the smart shoes: (a) a total of 12 sensors (6 sensors on the left and right shoe each) consisting of the pressure, accelerometer, and gyroscope sensors; (b) locations of the pressure sensors on the anatomical sketch: 1st metatarsal head (MH; sensor 1), toe (between the 1st and 2nd phalange; sensor 2), 4th metatarsal head (sensor 3), and heel (sensor 4).

• 

  Figure 3. Example figures of the experimental equipment and process. During the experiment, participants wore a chest strap and a calorimeter to measure HR and EE, respectively. Each participant ran on a treadmill at a speed varying from 3 to 10 kph, which increased by 1 kph per every 2 min (total 16 min ran) and they were instructed to run at a constant speed as much as possible.

• 

  Figure 4. Flowchart of the data preprocessing when training the proposed deep learning model. The input shoes’ data were recorded at a 33.3 Hz sampling rate and standardized to have a zero mean and unit variance. The label was created based on the HR and EE information, which were averaged on a 10 s long window with an overlap of 1 s.

• 

  Figure 5. Application of a linear interpolation method due to the mismatch between the sampling rates of the HR/EE and data of the shoes’ sensors. In the HR and EE graphs, the green dot represents HR and the gold dot represents the EE of the actual measurement, and the dashed line is the estimated value.

• 

  Figure 6. Distributions of HR and EE labels: (a,b) show the number of EE values per KCal/min and HR values per bpm, respectively.

• 

  Figure 8. Internal structure of DenseNet. ⊕ denotes the concatenation of feature vectors.

• 

  Figure 9. Structure of Bi-GRU.

• 

  Figure 10. Internal structure of GRU.

• 

  Figure 11. Comparison between the predicted (EST) and ground truths (REF) in EE estimation: (a) is the best case; (b) is the worst case.

• 

  Figure 12. Bland-Altman plot of EE estimation. The orange line is the limit of agreement (LOA) and the center blue line is the mean of difference error between the actual and estimation.