Answers are grounded in this paper and cite their sources.
This research aims to explore key issues in the field of walking robots and exoskeletons. By using a depth camera to record human walking data in different terrains and employing RGB and depth information for image classification, our approach utilizes both early fusion and late fusion methods to explore their impact on terrain classification. This approach achieved fast and accurate performance in recognizing terrain. We captured data for walking upstairs, downstairs, upward ramps, downward ramps and level ground and used the vision transformer for data fusion. This work compares the performance of unimodal RGB model, unimodal deep model, Pre-Fusion model and Post-Fusion model in terms of accuracy, recall and processing frequency, with accuracies of 0.954, 0.934, 0.976, and 0.976, respectively. The results show that the Post-Fusion model slightly outperforms the Pre-Fusion model in terms of accuracy and recall with more reliable classification performance. To understand the attention mechanism better, we've been visualizing the weights of the model, observing how different attention heads may capture different features of the data from various dimensional perspectives. The visualization helps in interpreting the model's focus and how it distinguishes between features of different nature, such as color texture and spatial depth. These findings can inform the ongoing refinement of terrain classification systems for applications in robotics and autonomous navigation.
No takes yet. Share an insight, caveat, or question.
Chen et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: