Bearing Remaining Useful Life Estimation Using Proximal Policy Optimization (PPO): Validation on the XJTU-SY Run-to-Failure Dataset

This study presents a proof-of-concept investigation into the use of proximal policy optimization (PPO), a deep reinforcement learning (DRL) algorithm, for estimating the remaining useful life (RUL) of rolling element bearings. Although DRL has shown growing promise in prognostics, existing applications have predominantly relied on off-policy deterministic actor–critic methods such as deep deterministic policy gradient (DDPG) and twin delayed DDPG (TD3); the suitability of on-policy clipped-objective methods such as PPO for this task remains comparatively unexplored. To address this gap, statistical time-domain features are extracted from raw vibration signals and used as input to train a PPO agent with an actor–critic architecture, in which the actor network predicts RUL values and the critic network evaluates prediction quality through state-value estimation. A preprocessing pipeline comprising feature extraction, normalization, and sliding-window segmentation is developed, and the PPO framework incorporates generalized advantage estimation (GAE), a custom-designed reward function, and a policy-clipping mechanism to support stable training. The method is evaluated on a representative bearing (Bearing 2_1) from the XJTU-SY run-to-failure dataset using a chronological train/test split, and benchmarked against long short-term memory (LSTM) networks, multilayer perceptrons (MLPs), and a naive linear regression baseline. Performance is assessed using root mean square error (RMSE), mean absolute error (MAE), mean squared error (MSE), and a domain-specific asymmetric scoring function that penalizes late predictions more heavily than early ones. Experimental results show that the PPO-based model produces more stable and operationally favourable RUL estimates than the supervised baselines on the unseen late-degradation segment, particularly in the critical end-of-life region. The findings support PPO as a viable on-policy DRL formulation for bearing RUL prediction and motivate further validation across multiple bearings and operating conditions, identified here as essential future work.