Robust Active Target Tracking via Constrained Policy Optimization in Partially Observable Environments

In recent years, active object tracking (AOT), which is a distinct technology from passive target tracking, has made significant advances in the field of artificial intelligence. AOT aims to actively track a target object by autonomously controlling the tracker's perspective and position when the target object leaves the current tracking scene. Previous research has demonstrated that trackers can be trained to perform active object tracking tasks using deep reinforcement learning. However, due to the partial observability and high-dimensional state space characteristics of active object tracking tasks, challenges arise during the training process, resulting in low sample efficiency and unstable model training, which increases the difficulty of model training. To address these challenges, this paper proposes a novel deep reinforcement learning framework based on constrained policy optimization, named CPO-AOT. This framework combines the asynchronous training mechanism with the concept of policy constraints. It not only accelerates training and reduces redundancy but also effectively constrains the magnitude of policy updates, ensuring that the agent does not make excessively aggressive policy changes. Additionally, for the active object tracking problem, CPO-AOT specifically designs a sparse reward function suitable for high-dimensional state spaces to guide the model's attention towards crucial states. Experimental results conducted in a three-dimensional virtual environment demonstrate that the adopted CPO-AOT framework enhances the model's performance in challenging environments with high-dimensional state spaces and partial observability. It effectively prevents the model from getting stuck in local optima, thereby strengthening the model's robustness. Compared to other methods, the proposed CPO-AOT proves more suitable for addressing active object tracking problems in partially observable environments.