Optimization of urban freight intelligent route based on reinforcement learning

Increased traffic congestion, limited delivery windows, vehicles' diverse fleets, and dynamic environments are all contributing factors to the inefficiency of urban freight systems. Due to the inability of traditional routing techniques to adjust in real-time to partially visible multi-constrained conditions, delivery delays, fuel consumption, and operating costs all increased. This research proposes a constraint-aware projected policy learning-reinforcement learning (CAPPL-RL) framework to optimize urban freight delivery routes while enforcing real-world constraints, including traffic congestion, delivery time windows, and vehicle capacity. The routing problem is formulated as a partially observable Markov decision process, with autonomous vehicles as agents navigating stochastic urban traffic networks. Using the UFVOD dataset, CAPPL-RL integrates Q-learning with ε-greedy exploration and projection-based constrained policy optimization (PCPO) to enable adaptive, constraint-aware routing. Simulation results demonstrate that CAPPL-RL outperforms PCPO-RL, reducing average delivery time from 65.3 to 52.1 min (20.2%), fuel consumption from 0.12 to 0.093 L/km (22.5%), and constraint violations in time windows from 12 to 3 (75%) while achieving 100% compliance with vehicle capacity limits. These results validate CAPPL-RL as a robust, scalable, and adaptive framework for dynamic urban freight logistics.