Athlete action quality assessment based on transfer neural network quality score decoupling in complex sports scenarios

Action quality assessment (AQA) is an important and challenging task in computer vision, which has received wide attention in many fields, especially in sports video analysis. In this paper, for the problem of uneven distribution of AQA scores in long sports videos, an AQA model based on transfer neural network quality score decoupling is proposed, which mainly consists of a dual-stream structure combining dynamic and static streams, a quality score decoupling module, and a pairwise sorting prediction module. Specifically, inspired by the action alignment processing in video understanding, a quality score decoupling module is designed based on the Transformer network decoder, which is able to decouple the input visual features into high/low quality score features, and at the same time adopts temporally average-pooled features as the average quality score representation. The overall skill level of a long video is assessed by focusing on the skill level related parts of the video in a pairwise order, and the task of assessing the quality of the action is accomplished by aligning the scores. In addition, the algorithm adopts a twin neural network structure to compare the input paired samples, and the dual-stream structure combining dynamic and static streams is able to extract the video motion information and frame-specific information separately, so that the model focuses on the dynamic time information and moment-specific action information simultaneously, which contributes to the feature extraction network to obtain a richer feature representation. This explicit separation of motion-centric and posture-centric representations avoids early entanglement of heterogeneous quality cues; the two streams are fused only after quality-aware feature disentanglement, following a late fusion strategy that is particularly suitable for long, untrimmed videos. Finally, by comparing with the existing methods on several public datasets, the comparison experiments and visual validation results show the effectiveness as well as the superiority of the proposed algorithm.