What question did this study set out to answer?

March 17, 2026Open Access

Problem-Related Performance Metrics of Deep Learning Models: Application to Swimmer Pose Estimation in Underwater Environments

Key Points

This work aims to establish a performance evaluation framework for underwater swimmer pose estimation using deep learning models.
Developed a scale-aware evaluation framework tailored for underwater imagery.
Introduced a dual assessment strategy combining anatomical tolerance thresholds with continuous NLA.
Assessed seven deep learning model configurations with varying training and processing strategies.
Achieved a Global Performance Index of 78.69 with models trained on diverse datasets.
Demonstrated performance gradients from core segments to distal extremities in swimmer poses.
Showed that conventional binary metrics inadequately represent the localization capabilities for small joints.

Abstract

• Scale-aware evaluation framework for underwater swimmer pose estimation • Continuous, tolerance-NLA for anatomical keypoints • Analysis of core-distal performance gradients in underwater imagery Accurate 2D swimmer pose estimation in underwater environments remains a challenging task due to optical distortions, dynamic occlusions, and the highly multi-scale nature of anatomical landmarks. Conventional evaluation metrics adopted from generic computer vision benchmarks are often inadequate to characterise the functional reliability required in sports biomechanics, particularly when small and fast-moving joints are involved. This work proposes a problem-orientated evaluation framework for underwater swimmer pose estimation, applied to seven deep learning model configurations differing in training data composition, pre-processing strategies, and parameter optimisation. Beyond the indicators based on standard confusion-matrix, a dual assessment strategy is introduced, combining strict anatomical tolerance thresholds with a continuous tolerance-Normalised Localisation Accuracy (NLA). Keypoint-specific tolerances are derived from the spatial extent of each anatomical region, allowing scale-consistent evaluation throughout the kinematic chain. Experimental results show a pronounced performance gradient from core body segments to distal extremities, highlighting the limitations of binary metrics for small joints. Models trained on heterogeneous raw datasets achieve the best overall performance (Global Performance Index = 78.69), demonstrating superior robustness and generalisation. Comparative analysis reveals that binary tolerance-based metrics are overly punitive for distal landmarks and tend to obscure the true localisation capability of the models. The proposed continuous NLA provides a more informative representation of spatial uncertainty and measurement quality. These findings emphasise the importance of problem-related, scale-aware evaluation metrics and confirm data diversity as a more effective driver of robustness than aggressive pre-processing in underwater swimmer pose estimation.

Problem-Related Performance Metrics of Deep Learning Models: Application to Swimmer Pose Estimation in Underwater Environments

Key Points

Abstract

Cite This Study