A World Model in 41k Parameters: Planning and On-Chip Learning with a JEPA on an ESP32-S3

Joint-Embedding Predictive Architectures (JEPAs) learn world models that predict the future in a compact latent space rather than in pixels, which makes them candidates for planning on very small hardware. This paper tests whether such a model can run, plan and keep learning on a microcontroller. TinyJEPA is an action-conditioned JEPA with 41,217 deployed parameters that fuses a 96 × 96 grayscale camera, four sonars and an inertial measurement unit. It runs in 8-bit integer (INT8) arithmetic on an ESP32-S3 microcontroller and plans with model predictive path integral (MPPI) control in its latent space, while the chip drives a simulated mecanum-wheel robot in Webots over a serial link. The evaluation consists of seven gates, whose criteria and analysis scripts were fixed before each run. Three results are positive. On the chip, the INT8 model reproduced the PyTorch model and three bit-exact optimisations reduced encoder latency from 70.7 to 16.8 milliseconds, so that a complete planning decision took 266 milliseconds. In the loop, the chip was statistically equivalent to the PC planner on collision and goal rates. Adapting the predictor on the second core, concurrently with control, lowered one-step prediction error by 11.6% after a change in the robot's dynamics (p < 0.0001), without forgetting, at 2.24 learning steps per control step. Three results are negative or qualified. The original budget of one decision per 96-millisecond control step was not met, so the system decides every 288 milliseconds. On a sealed test of 100 maps, the planner did not collide significantly less than a tuned reactive sonar controller (53% versus 60%, p = 0.094) and reached fewer goals (13% versus 30%). It delayed collisions rather than avoiding them, mainly because collision prediction is weak outside the camera's field of view. Finally, the change in dynamics raised the frozen model's error by 1.197 times against a required 1.2, so the adaptation result is formally inconclusive. All gates, amendments, code, firmware and data are released.