Merits of curiosity: a simulation study

'Why are we curious?' has been among the central puzzles of neuroscience and psychology in the past decades. Recent 'top-down' theories have hypothesized that curiosity, as a desire for some intrinsically generated rewards (e.g., novelty), is the optimal solution for survival in complex environments where we have evolved. To formalize and test this hypothesis, however, it is necessary to understand the relationship between (i) intrinsic rewards (as drives of curiosity), (ii) optimality conditions (as objectives of curiosity), and (iii) environment structures. Here, we demystify this relationship through a systematic simulation study. We first propose an algorithm for generating environments that capture key abstract features of different real-world situations. Then, within these environments, we simulate different artificial agents seeking six representative intrinsic rewards (novelty, surprise, information gain, empowerment, MOP and SPIE) and evaluate their performance regarding three potential objectives of curiosity (environment exploration, model accuracy and uniform state visitation). Our results show that the comparative performance of each intrinsic reward is highly dependent on the structural features of environments and the objective under consideration; this indicates that 'optimality' in the top-down theories of curiosity needs a precise formulation of the curiosity objective and the environment structure. Nevertheless, we found that agents seeking a combination of novelty and information gain always achieve a close-to-optimal performance; this proposes novelty and information gain as two principal axes of curiosity-driven behavior. These results, collectively, pave the way for the further development of computational models of curiosity and design of theory-informed experimental paradigms.