Skeleton-Based Traffic Gesture Recognition via a Motion-Guided 2S-GCN in Autonomous Driving

For the safety of autonomous driving systems, recognizing the gestures of cyclists and controllers on traffic roads is crucial. However, general action recognition methods are unsuitable for traffic gestures. This is because, compared to general actions, traffic gestures exhibit distinct characteristics, namely:(1) short action duration; (2) atypical action posture; and (3) small dataset sizes. We propose a novel two-stream graph convolutional network (2S-GCN) to address these challenges. Firstly, we introduced a motion-guided module connecting the motion and joint stream. By leveraging the motion stream’s spatial expression capability, we guided the joint stream’s learning in spatial dimensions to enhance the network’s performance to short action durations. Secondly, we utilized a simplified skeleton topology that exclusively captures the upper body during data preprocessing, thereby preserving essential information while eliminating redundancy. This approach enhances the network’s adaptability to atypical action postures. Thirdly, we use a GCN embedded with a multi-channel attention module as the backbone, which is particularly suitable for small datasets, and conduct extensive experiments on two datasets. The experimental results demonstrate that our network can accurately recognize traffic gestures and has significant advantages over state-of-theart methods.