How to Build an Observatory for the Mind

Psychological theories are rarely explicitly rejected or gradually incorporated into a cumulative theoretical structure. Instead, they rise and fall with the whims of individual researchers. Formal modeling offers a potential solution to this crisis by strengthening the link between theories, hypotheses, and data, and are becoming increasingly popular in many subfields of the experimental behavioral sciences. However, evaluating these models poses its own set of challenges. Current methods often rely on small, ad-hoc experiments that lack the statistical power, generality and diversity needed to robustly test these models. To solve these issues, we propose to create an Observatory of the Mind – an independent scientific institute dedicated to the systematic collection and analysis of experimental behavioral data. The institute will maintain a large pool of long-term semi-professional participants, representing a cross-section of the global population, who will complete a comprehensive set of empirical benchmark tasks for behavioral science over several years. The goal is to produce a publicly available body of knowledge that can support the development and evaluation of formal models of psychological processes and enable algorithmic discovery of novel theories. We argue that such an observatory is necessary to overcome the current crisis in psychological research, characterized by weak theories, unreliable findings, and low generalizability. We also discuss the practical benefits and the logistical challenges of implementing this project. We invite the scientific community to engage in a constructive dialogue about the feasibility and potential impact of such an observatory.