Decoupling Geometric Planning From Physical Interaction for Multi-Category Peg-in-Hole Assembly

Ensuring reliable contact execution across different peg-in-hole assembly tasks remains a key challenge in robotic assembly. This paper proposes a hierarchical learning framework for multi-category peg-in-hole assembly that functionally separates long-horizon geometric motion generation from contact-level physical adaptation. A task-conditioned multimodal diffusion policy learns a shared geometric prior from multi-task demonstrations and generates smooth, geometrically feasible motion trajectories. During physical interaction, a contact-triggered residual adapter trained via force-constrained reinforcement learning performs bounded local pose correction and stiffness modulation. This design preserves cross-task reuse of the geometric prior while localizing online contact adaptation to residual adapters trained separately for each task. Real-robot experiments across nine geometric and connector insertion tasks show that the proposed method improves assembly success and interaction robustness over learning-based and control-based baselines, while component ablations and sensitivity analyses further clarify its practical scope.