MARL-PINN: Solving the Eikonal equation with multi-agent reinforcement learning method for physics-informed neural networks

The initial arrival time of seismic waves serves as crucial data in seismic information, efficiently depicting underground velocity structures. The Eikonal equation, a nonlinear partial differential equation utilized for modeling seismic first-arrival travel-times, illustrates how seismic waves propagate in heterogeneous 3D velocity environments. Recent research has shown that Physics-Informed Neural Networks (PINN) have successfully solved specific partial differential equations. However, when applied to the Eikonal equation, PINN exhibit error accumulation that increases with propagation distance. This limitation affects their ability to accurately model seismic wave travel times over longer distances, highlighting the need for improved methodologies to enhance the precision of seismic data interpretation. To address this limitation and the sensitivity to loss function weighting in PINN, we propose a novel multi-agent reinforcement learning (MARL) enhanced PINN approach for solving the Eikonal equation. Our method introduces two agents that dynamically adjust the loss weights (specifically, the weights for the partial differential equation loss and the boundary condition loss) during training. Each agent employs a Deep Q-Network (DQN) algorithm to update its policy, aiming to minimize the overall loss. We rigorously tested our approach on 3D velocity models to validate its robustness and generalizability. Comparative evaluations on conventional geological velocity models demonstrate that our method achieves higher accuracy than standard PINN approaches while effectively mitigating the error accumulation observed with increasing distance from the source.