Optimization-based Imitation Learning for Stable Action Generation

Imitation learning is widely used to learn robotic manipulation behaviors from expert demonstrations, but conventional behavior cloning primarily minimizes the discrepancy between demonstrated and predicted actions. As a result, task-oriented objectives and execution constraints may not be explicitly reflected in the learned policy, and small prediction errors may lead to task failure in states that are insufficiently covered by the demonstrations. This paper proposes Optimization-Based Imitation Learning (OB-IL), which combines a pretrained nominal policy, a lightweight residual prediction head, and residual quadratic programming (QP). The nominal ACT policy is initially trained from expert demonstrations and then frozen to preserve the learned task behavior. During policy rollouts, the residual QP computes a bounded corrective action that reduces a task residual while limiting deviations from the reference action. The QP-generated residual correction is used as supervision for the residual prediction head, which predicts corrections from the frozen ACT feature representation. During inference, the learned residual is combined with the ACT output and the resulting action is further refined using the residual QP. In MuJoCo peg-in-hole experiments, the nominal ACT policy achieved a 9.0% success rate with a median lateral alignment error of 6.43 mm, whereas OB-IL achieved an 89.0% success rate and reduced the median error to 0.85 mm. These results support the effectiveness of the proposed approach for the evaluated precision insertion task; extension to other manipulation tasks remains to be validated experimentally.