HHANet: a human hand acupoint key point detection network based on multi-scale fusion and transformer

In Traditional Chinese Medicine diagnostic and rehabilitation training, hand acupoint keypoint detection offers technical assistance for accurate placement and objective quantification. Predicting the keypoints of small objects, like the hand, is a difficult problem. In order to provide a crucial technology for hand acupoint keypoint identification, feature fusion in deep convolutional neural networks has concentrated on combining multi-level semantic information with multi-scale positional information in recent years. Keypoint detections may be overlooked due to the fixed sampling areas and single contextual information of feature maps of various dimensions, despite the fact that these methods have had some success. We present a hand keypoint detection technique (HHANet) that combines transformers and multi-scale fusion to address these problems. In order to balance global information processing with local feature extraction, we first construct a Multi-Transformer to increase the model's anti-interference performance and its comprehension of global information. While maintaining the Transformer's capacity to describe long-range dependencies, the Multi-Transformer is able to better capture spatial features by incorporating a convolutional structure within a local window. Second, although the Transformer module improves the backbone network's feature representation capabilities, it still has issues with edge keypoint detection. In order to solve this problem, we created a low-frequency filter enhancement module (LEF) that can record edge acupoint texture. Lastly, we suggested a cascaded hybrid attention mechanism (FEM) to increase the accuracy of acupoint identification in the presence of extreme noise and interference. In contrast to current hybrid attention, FEM uses a stepwise augmentation process, initially channel-wise and later spatially, to highlight fine-grained keypoint features. Experimental results using a self-developed HHA dataset show that the suggested network strikes a decent balance between detection performance and model complexity when compared to state-of-the-art techniques.