Preference‐Based Representations for Collective Agency

Computational accounts of learning and decision-making in cognitive systems, and models thereof, such as reinforcement learning, typically assume that the behavior of individual agents is determined by an externally designated reward signal such that the agent's goal is the maximization of its expected accumulated reward under the state dynamics of the world. Such frameworks, therefore, struggle to explain how coordinated behavior emerges in collective contexts involving multiple agents with potentially shared goals. To address this gap, we draw on a recently proposed computational framework of preference-based representations in cognitive agents. We extend this framework from a single- to a multiagent setting to explain how coordinated behavior can emerge from agents' subjective preferences over self- or other-oriented simulated experiences. We argue that such experience simulation mechanisms offer a cognitively plausible account for describing the acquisition, refinement, and sharing of goal-directed representations in cognitively bounded agents. We situate our proposed framework within current debates in artificial intelligence and cognitive science concerning notions of individual and collective agency.