Answers are grounded in this paper and cite their sources.
Three-dimensional (3D) pose estimators lose joints whenever a limb is hidden, yet learned reconstruction of missing joints is usually developed with simulated occlusion, because the true positions of occluded joints are rarely known. This study evaluates reconstruction under real occlusion with measured ground truth: in 3D Poses in the Wild (3DPW), a joint is lost when its OpenPose confidence falls below 0.3, while body-worn inertial sensors give its true position; CMU Panoptic, Human3.6M, and MPI-INF-3DHP serve as zero-shot tests. The benchmark therefore scores the joints that a real detector misses in real video, which is what a deployed system has to fill, rather than joints annotated as physically occluded. Thirteen models were compared with five seeds and a paired video-level bootstrap, including the proposed Occlusion-Aware Skeleton Spatiotemporal Transformer (OA-Skeleton ST), which starts from linear interpolation and learns a visibility-aware residual correction, and which is pretrained on AMASS motion capture with simulated masks before fine-tuning on 3DPW. On the joints really lost in 24 test videos, OA-Skeleton ST reached the lowest mean per-joint position error, at 79.1 mm: 13.7 mm below interpolation with a kinematic fallback (95% CI 9.0–17.6 mm, better on 87.5% of videos) and 6.7 mm below a plain spatiotemporal transformer (95% CI 4.9–8.7 mm, better on every video). Removing the AMASS pretraining stage cost 14.8 mm (95% CI 8.3–21.4 mm), making it the largest single contributor. The advantage persisted out of domain, where the occlusion-aware models reached 132.7–145.5 mm on CMU Panoptic, 132.4–144.5 mm on Human3.6M, and 117.8–118.7 mm on MPI-INF-3DHP, against 152.9, 163.0, and 155.4 mm for the heuristic; the variant that keeps the kinematic anchor, which is neutral in domain, transferred about 12 mm better. Joints never seen within the one-second window remained hardest at 119.7–146.1 mm, and simulated masks preserved the model ranking (Spearman 0.93) but overstated the advantage of learning threefold. These figures were obtained with the measured positions of the visible joints, so they quantify imputation rather than deployment: when the visible joints are themselves estimated by a monocular pipeline (OpenPose to VideoPose3D), every method lands between 268 and 300 mm, the structural heuristic and the learned models are no longer separated, and only a model trained on the pipeline’s own output (267.8 mm) improves on them. Learned spatiotemporal attention improves on a strong structural prior under real occlusion, provided it is initialised from that prior and pretrained on large-scale motion capture.
No takes yet. Share an insight, caveat, or question.
Kibet et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: