OP‐Follower: Onboard Vision‐Based Obstacle‐Aware Predictive Person Following for Quadruped Robots via IBVS‐NMPC Control

This paper presents OP‐Follower , a fully onboard, pure vision‐based, obstacle‐aware person‐following framework for quadruped robots. Unlike existing approaches that rely on LiDAR, infrastructure, or offboard computation, OP‐Follower operates exclusively on visual inputs and runs entirely on a Unitree Go2. OP‐Follower bridges the gap between low‐frequency, computation delay‐prone visual perception and safety‐aware high‐rate control by integrating YOLO‐World–based open‐vocabulary person detection, BoxMOT multi‐object tracking, and Kalman filter–based delay‐aware prediction within a unified planning–control architecture that tightly couples image‐based visual servoing with nonlinear model predictive control. A decoupled artificial potential field–based visual obstacle avoidance strategy is further employed to ensure collision‐aware navigation without interfering with target tracking objectives. Comprehensive evaluations in high‐fidelity Gazebo simulations and real‐world indoor and outdoor environments, including long‐distance following, dynamic pursuit–evasion interactions, and multi‐person scenarios, demonstrate robust real‐time performance with fully onboard processing at a 50 Hz control rate. Compared with a non‐predictive baseline, OP‐Follower reduces pixel‐space and distance tracking errors by up to 57.2% and 50.5%, achieving representative root‐mean‐square errors of approximately 13–16 pixels and 0.09–0.11 m under sharp turning motions, while maintaining stable outdoor performance, including a continuous 1.4 km trial. GitHub page with demo videos is available at https://op-follower.github.io/ .