Model-based planning in structured foraging environments

In order to maximize reward, humans need to balance engaging with currently available sources of reward and searching for better ones. Optimal foraging theory provides a formal but simple mathematical choice rule to make such stay/leave decisions, contrasting expected and experienced rewards. However, this rule (given by the Marginal Value Theorem; MVT) describes a strategy that does not consider the structure of the environment. In other words, it does not leave room for planning during foraging. Yet, the real world is replete with such opportunities. Therefore, we developed a new structured foraging task to study how people employ goal-directed planning during foraging. Specifically, we explore the extent to which participants incorporate an internal model of the task structure during stay/leave decisions. We find that behavior in this task follows the basic principles of the MVT, but that its structure invites people to also consider the value of alternative reward options when deciding to leave their current one. Importantly, this behavior is pronounced in more goal-directed participants. Computational modeling suggests that incorporating this alternative information is beneficial, but to an extent dictated by choice stochasticity. This study provides a novel method for studying decision making in structured environments, and has implications for understanding how foraging and planning interact.