Ventilator Treatment Policy Control based on BCQ off-line Deep Reinforcement Learning

Abstract Ventilator plays a crucial role in treating cardiopulmonary disorders, and proper parameter settings are essential for the successful recovery of patients. Traditional ventilator control relies on the expertise of physicians, leading to delayed treatment responses. Although some machine learning methods have made improvements in this scenario, they are inadequate to adapt to dynamic changes of patient conditions. This paper proposes a dynamic ventilator control method based on the BCQ offline deep reinforcement learning algorithm, achieving real-time adjustment of ventilator treatment policies based on changes in the patient’s medical condition. In the experiments, the Double DQN and SAC algorithms are used as baseline algorithms. During the training phase, the algorithms’ optimal models under different hyperparameter combinations are determined using temporal difference error and average action values. In the testing phase, the model’s therapeutic efficacy is evaluated using the FQE method, while the safety of the treatment is assessed by statistically analyzing the predicted action distribution. Additionally, the algorithm’s generalization ability is further evaluated on an OOD test set. The experimental results demonstrate that the BCQ algorithm outperforms both in terms of treatment effectiveness, safety, and generalization ability, indicating its promising application prospects in medical scenarios.