Deep Reinforcement Learning for Multi-Objective Optimization of Latency-Constrained V2X Communications

Meeting the stringent latency requirements of safety-critical vehicular services is a major challenge for beyond fifth generation (B5G) systems, and rapidly varying channel state information (CSI) in device-to-device (D2D) enabled vehicle-to-everything (V2X) communication makes the problem harder. This paper addresses power allocation in a D2D-enabled vehicular network in which vehicle-to-vehicle (V2V) links reuse the spectrum of vehicle-to-infrastructure (V2I) links. The problem is formulated as a non-convex optimization problem that maximizes the V2I sum capacity subject to a V2V latency constraint. We use deep reinforcement learning to jointly select the V2I and V2V transmit powers from large-scale channel information, trained from measured outcomes rather than from the closed-form channel expressions used by the optimization-based solution, and a normalized preference weight that trades V2I capacity against V2V latency within a single network. The framework is implemented with a discrete-action deep Q-network (DQN) and with a continuous-action deep deterministic policy gradient (DDPG) agent. Under the nominal channel model, the DQN attains 85–95% and the DDPG agent 90–99% of the sum capacity of the model-based optimum, with a per-block computation time two to three orders of magnitude lower. When the reported large-scale channel gains contain errors of 2 dB, the model-based solution violates the V2V latency constraint on 46–56% of the links, whereas agents trained under such errors keep the violation rate below 0.5%. DDPG is preferable when the channel model is accurate, whereas the DQN trains more reliably under CSI errors. Queue-level simulations further show that, at the considered traffic loads, queueing rather than transmission failures dominates the delay tail.