An Exploratory Reinforcement Learning Simulation Framework for Studying Antimicrobial Resistance Dynamics Under Copper Exposure

This study presents an exploratory reinforcement learning (RL)-based simulation framework for examining antimicrobial resistance (AMR) dynamics under repeated exposure to a non-antibiotic stressor, using copper as a simplified model compound. The objective is not to provide mechanistic or predictive insight into microbial evolution, but to evaluate how alternative sequential decision-making strategies perform within a constrained and transparent simulation environment. Three agent strategies were compared: random action selection, a rule-based heuristic, and a tabular Q-learning agent. Simulations were conducted over fixed 40-cycle episodes in which agents adjusted copper exposure in response to evolving resistance-related state variables. Across experimental runs, the Q-learning agent exhibited lower cumulative resistance burden, measured by area under the curve (AUC) of minimum inhibitory concentration (MIC) trajectories for chloramphenicol and polymyxin B, while maintaining lower cumulative copper exposure relative to baseline strategies. The rule-based agent demonstrated intermediate performance, whereas the random agent showed greater variability and less stable trajectories. These findings reflect differences in simulated control behavior within a simplified stochastic system. Overall, this work introduces an interpretable reinforcement learning simulation tool intended to support comparative evaluation of adaptive versus static strategies in antimicrobial pressure management under limited observability.