EfRaRL: Embedded Failsafe Robust Actor Reinforcement Learning for Resource-Constrained Microcontrollers

Deep Reinforcement Learning (DRL) has demonstrated remarkable success in continuous control tasks. However, its deployment on resource-constrained microcontrollers (MCUs) remains a significant challenge due to extreme memory limitations and the absence of dynamic memory allocation (malloc). Most existing literature focuses solely on on-device inference by training models offline on powerful machines and transferring the frozen weights to the edge. In this paper, we introduce EfRaRL (Embedded Failsafe Robust Actor Reinforcement Learning), a zero-allocation, Continuous Deep Deterministic Policy Gradient (DDPG) framework designed specifically for Cortex-M class microcontrollers. EfRaRL natively operates within a 16.0 KB static memory footprint, achieving full on-device training capability. By employing a double-buffering architecture, the framework guarantees Interrupt Service Routine (ISR) safety, preventing thread-locking in real-time robotic control. We evaluate EfRaRL under an adversarial hardware-in-the-loop simulation featuring severe sensor noise, dropouts, and physical latency, and compare its performance against state-of-the-art frameworks. Hardware-in-the-loop tests on an STM32 microcontroller successfully validate its real-world efficacy.