Detecting Failures in Robot Manipulation via Distance‐Aware Temporal Modelling

ABSTRACT Failure detection is essential for ensuring the safety of robotic manipulation policies, yet modern imitation learning methods, such as diffusion‐based policies, remain vulnerable to subtle trajectory deviations that are not visually apparent. We propose a distance‐aware failure detection framework that models the temporal evolution of robot states and object‐relative distances using a lightweight residual‐based model and a stochastic consistency score, along with a rollout‐calibrated temporal prediction band to account for discrepancies between demonstrations and policy rollouts. Experiments on simulation manipulation benchmarks show that our method is effective when visual differences between successful and failed executions are subtle, demonstrating the benefit of distance‐aware failure detection.