A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth

Stable three-dimensional hand landmark reconstruction using low-cost RGB-D sensors is important for human–computer interaction, robot teleoperation, and vision-based motion analysis. RGB-based hand landmark detectors provide stable semantic 2D landmarks, but their depth output is not a metric measurement in the physical camera coordinate system. Stereo cameras can provide metric depth, but direct landmark-level back-projection is sensitive to invalid pixels, local depth holes, boundary noise, and partial occlusion. To address these problems, this paper presents a lightweight RGB-D sensing front-end that combines MediaPipe semantic hand landmarks with ZED2 stereo depth. The proposed pipeline detects 21 semantic hand landmarks in the RGB image, obtains landmark-level metric depth from the aligned ZED2 depth map using local median sampling, reconstructs 3D landmarks by camera back-projection, and further applies exponential moving average filtering and a bone-length consistency constraint. Experiments were conducted on a self-collected SVO dataset containing 13 hand actions and 26 recorded sequences, and an additional checkerboard-based reference-distance validation was performed to evaluate the metric depth sampling and 3D back-projection component. Compared with single-pixel sampling, the 5×5 local median strategy slightly increased the valid-depth ratio from 0.9731 to 0.9738 and reduced the temporal smoothness metric from 1.7163 mm to 1.6902 mm. To further justify the temporal filtering choice, an additional comparison with the 1 Euro Filter was conducted using the reconstructed win5 trajectories. The 1 Euro Filter produced stronger smoothing, reducing the temporal smoothness metric to 0.196 mm, but also reduced the path-length ratio to 0.484, indicating substantial motion attenuation. EMA0.7 was therefore retained as a more balanced setting, reducing the temporal smoothness metric to 0.826 mm while maintaining a path-length ratio of 0.803. The BL0.5 bone-length constraint reduced the bone-length standard deviation from 2.0727 mm to 1.1995 mm with limited trajectory modification. The final configuration provides a practical low-cost RGB-D front-end for stable 3D hand landmark reconstruction under controlled indoor conditions.