A naturalistic reinforcement learning task uncovers historical neural representations

Exploration is essential for reinforcement learning (RL) in human development, facilitating cognitive and behavioral adaptation. However, conventional RL paradigms in neuroimaging studies often rely on highly constrained search spaces and artificial, non-naturalistic scenarios, limiting their capacity to reflect the complexity of real-world human exploration and decision-making. To address this gap, we developed a novel, naturalistic RL paradigm called Photographer, which involves a photograph-taking task set in a virtual street-view environment. Participants navigated city scenes and attempted to infer the paradigm's covert goal solely from feedback, defined as the conceptual similarity between their photograph and a hidden target sentence. Representational similarity analysis revealed that feedback history, an encoding of recent trial attempts, was robustly represented in the middle orbital gyrus and inferior frontal gyrus. These neural representations were consistently observed across two independent participant groups. Moreover, although the two regions coactivated during exploration, each was associated with a distinct functional network.