A transductive learning-based method for vehicle routing problems using off-policy proximal policy optimization and hyperparameter optimization

The Vehicle Routing Problem (VRP) is a fundamental combinatorial optimization task that involves determining cost-effective routes subject to operational constraints. This study proposes a deep reinforcement learning framework that integrates Off-policy Proximal Policy Optimization (PPO) with a transductive LSTM (TLSTM), further enhanced by a differential evolution-based hyperparameter optimization (HO) procedure. The TLSTM module captures spatiotemporal dependencies more effectively than conventional LSTMs, while the off-policy PPO component enables robust policy updates under diverse VRP variants. The HO module ensures stable convergence by adaptively balancing exploration and exploitation. The proposed approach is evaluated on five VRP variants—including the Traveling Salesman Problem (TSP), Capacitated VRP (CVRP), Split Delivery VRP (SDVRP), Orienteering Problem (OP), and Prize Collecting TSP (PCTSP)—demonstrating consistent improvements in both solution quality and computational efficiency. On average, the framework achieves a 4.12% reduction in route costs and a 94.34% improvement in computational speed, underscoring its potential for advancing VRP research and supporting practical applications in logistics and transportation.