Learning to Nudge:Affect-Aware Model-Free Reinforcement Learning for Energy Conservation

Digital energy feedback systems often present static, uniform messages that ignore users’ emotional context, reducing long-term effectiveness. We propose an affect-aware, model-free reinforcement learning framework for personalized energy-feedback nudging. The system extends the classical MAPE-K loop with an Affect–Behavior Decoupling Architecture (ABDA) that processes emotional signals (via real-time facial emotion recognition) and behavioral cues in parallel. By learning when, how, and which nudges to deliver based on the user’s detected affective state and past interaction data, the agent autonomously tailors interventions without explicit pre-programmed rules. We implemented this approach in a simulated smart-home dashboard and conducted a controlled laboratory study. Results show that emotionally adaptive feedback led to higher positive affect and sustained engagement compared to a non-adaptive baseline. The adaptive condition also increased exposure to conservation-relevant cues and produced modest gains in self-reported energy awareness. We position household energy conservation as the application context. Demonstrating direct impact on energy consumption requires longitudinal field studies. The paper contributes with i) an ABDA–MAPE-K design for hybrid affective-behavioral adaptation; ii) a model-free reinforcement-learning agent that discovers personalized nudge policies from data; and iii) an empirical evaluation demonstrating the benefits of affect-sensitive adaptation for engagement and conservation-relevant cueing.