Answers are grounded in this paper and cite their sources.
Learning dexterous manipulation typically requires hundreds of demonstrations, making data collection costly and hard to scale. We propose a three-stage framework learning from a single human video. First, hand and object trajectories extracted from the video are retargeted to the robot as motion priors. Second, combining residual learning with trajectory augmentation, a privileged state policy is trained via proximal policy optimization to generalize to unseen object poses. Third, a diffusion model distills this policy into a visual controller: the privileged policy collects self-generated successful trajectories under standard and perceptual domain randomization conditions, which are blended to train the diffusion model to output action sequences from raw point clouds. Across three dexterous manipulation tasks, the privileged state policy attains high success rates with one demonstration video per object, achieving higher SR10 success rates than DexMV baselines on the same Adroit platform with two orders of magnitude fewer demonstrations, while the distilled vision-based policy operates from raw point clouds without privileged state information. This work demonstrates that effective dexterous manipulation strategies can be learned even when human supervision is limited to a single demonstration video. The video supplies the motion prior, while all the remaining training experience, including the robot rollouts that are used for distillation, is generated autonomously in simulation. This offers a practical pathway for low-cost robotic skill learning.
No takes yet. Share an insight, caveat, or question.
Huang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: