A Real-Time Deep Reinforcement Learning Processor for Mapless Autonomous Navigation With Unified Actor-Critic Network and Inference-on-Request Scheduling

This article presents a real-time deep reinforcement learning (DRL) processor for mapless autonomous navigation targeting resource- and energy-constrained mobile robots. The unified actor-critic network architecture combined with feature map caching enables parameter sharing and eliminates redundant computations. This approach reduces the total parameter count, external memory access (EMA), and overall operations by 95.6%, 84.1%, and 85.4%, respectively, without compromising learning performance. The processor further incorporates an inference-on-request (IoR) scheduling scheme to achieve concurrent training and inference, thereby ensuring low inference latency and maintaining high processing element utilization of 71.0%, including external I/O overhead. A column-wise zero-skipping categorical projection accelerates distributional reinforcement learning by processing only non-zero entries in the sparseΔ hmatrix, achieving a 99% reduction in projection operations. Fabricated in a 28-nm CMOS process, the chip consumes 2.68 mW with a 10-MHz clock frequency, meeting the real-time latency requirements. The processor was verified on a real robot, demonstrating robust autonomous navigation ability in unseen environments. To the best of our knowledge, this is the first design to support distributional reinforcement learning, and the experimental results with a real robot confirm both the effectiveness and practical viability of the design.