A computational model of behavioral adaptation to solve the credit assignment problem

Abstract The adaptive fitness of an organism in its ecological niche is highly reliant upon its ability to associate environmental or internal stimuli with behavioral responses through reinforcement. This simple but powerful approach has been successfully applied in computational neuroscience and reinforcement learning to model both human and animal behaviors. However, a critical challenge faced by these models is the credit assignment problem, in which the association between past behavior and a delayed reinforcement signal must be considered. In this paper, we reformulate the credit assignment problem to consider how past stimuli are linked to adaptive behavioral responses in a simple neuronal circuit. We propose a biologically plausible variant of a spiking neural network, which can model a wide variety of behavioral, learning, and evolutionary phenomena. Our model suggests one fundamental mechanism for associating a behavior with an adaptive response that may be used in the brains of both simple and complex organisms. Our results show the model's versatility and biological plausibility in a number of tasks related to classical and operant conditioning, including behavioral chaining. Additionally, we present simulations to demonstrate how adaptive behaviors such as reflexes and simple category detection may have evolved using our model. Our results indicate the potential for further modifications and extensions of our model to replicate more sophisticated and biologically plausible behavioral, learning, and intelligence phenomena found throughout the animal kingdom.