Patient-Specific Deep Reinforcement Learning for Proton Beam Delivery Under Inter-Phase Variations

Purpose: Proton therapy is challenged by tumor motion, particularly for lung tumors affected by respiratory-induced motion. Conventional planning strategies compensate for this motion by introducing safety margins or by using robust optimization, increasing irradiation of surrounding healthy tissues. Real-time plan adaptation during delivery represents a promising alternative to mitigate intrafractional motion effects. Materials and Methods: We propose a patient-specific deep Reinforcement Learning (RL)-based control framework for proton pencil beam scanning, formulated as a first step toward a real-time plan adaptation problem under respiratory motion. RL agents are trained independently on 3 patients on their mid-position planning CT to sequentially control beam position and spot delivery. At inference, the learned policy is executed on the respiratory phases of each patient's 4DCT without any online re-training or model parameters adaptation. The agent is provided with 2D observations encoding target geometry, beam position, and prior spot delivery. The approach is evaluated against conventional static gross tumor volume-based (GTV-based) and internal target volume-based (ITV-based) planning strategies. Results: for Patient 3, who has the largest motion amplitude. Conclusion: This study should be interpreted as a proof-of-concept and highlights the potential of RL-based control strategies for proton therapy delivery under intrafractional motion. While the current framework relies on static training, it establishes a foundation for future extensions toward fully dynamic and adaptive treatments.