Decoding Human Prediction Errors for Human-Robot Value Alignment

In human-robot interaction, obtaining value alignment is crucial for enhancing working efficiency. Typically, efforts are made to decode human goals from their actions. One critical issue is that action-goal mapping is not injective; e.g., one action can be associated with different goals. This inherent complexity underscores the need for innovative approaches to enhance the collaborative capabilities of robots. Humans evaluate one's or others' actions based on the difference between expectations and actual observations with respect to situations (state) and outcomes (rewards), each of which is called state prediction error (SPE) and reward prediction error (RPE) under the context of human reinforcement learning (RL). Here, we hypothesize that leveraging these key variables in human RL—SPE and RPE—via brain-computer interface (BCI) can attain value alignment at both action and goal levels. To demonstrate that SPE and RPE could be dissociated from human electroencephalography (EEG) data, we conducted experiments inducing different prediction errors in a GridWorld environment with a navigating robot. The significant variations between the SPE and RPE decoding accuracy across diverse frequency bands and distinct brain regions illustrated the dissociability of human prediction errors. We believe it is achievable to implement our strategy to BCI-guided human-robot alignment framework.