DynamicRL: Data-Driven Estimation of Trial-by-Trial Reinforcement Learning Parameters

In uncertain and dynamic environments, biological agents must adapt their decision-making strategies to maximize rewards. Traditional reinforcement learning (RL) models typically assume that such adaptation is governed by dynamic value updates controlled by fixed parameters or predefined schedules. However, these assumptions limit the models' ability to capture the flexible and context-sensitive nature of biological decision-making. To overcome this limitation, we introduce DynamicRL, a novel framework that estimates RL parameters from behavioral data on a trial-by-trial basis. We demonstrate that DynamicRL substantially improves the predictive performance of standard RL models across eight decision-making tasks, thereby reducing scientific regret. DynamicRL captures the rich temporal variability inherent in decision-making behavior, achieving predictive performance comparable to that of recurrent neural networks trained directly on the data, while preserving the interpretability and theoretical grounding of RL models. Moreover, it enables the examination of how agents dynamically adjust RL parameters in response to environmental changes, offering insights into the cognitive mechanisms underlying such adaptations. Thus, DynamicRL serves as an efficient data-driven framework for estimating RL parameters, facilitating fine-grained behavioral analysis with potential applications in computational psychiatry and neuroscience.