Combining Active Inference and Hierarchical Predictive Coding: A Tutorial Introduction and Case Study

This paper combines the active inference formulation of action (Friston, 2009) with hierarchical predictive coding models (Friston, 2003) to provide a proof-of-concept implementation of an active inference agent able to solve a common reinforcement learning baseline -- the cart-pole environment in OpenAI gym. It demonstrates empirically that predictive coding and active inference approaches can be successfully scaled up to tasks more challenging than the mountain car (Friston 2009, 2012). We show that hierarchical predictive coding models can be learned from scratch during the task, and can successfully drive action selection via active inference. To our knowledge, it is the first implemented active inference agent to combine active inference with a hierarchical predictive coding perceptual model. We also provide a tutorial walk-through of the free-energy principle, hierarchical predictive coding, and active inference, including an in-depth derivation of our agent.