DI-MATD3-Based Multi-UAV Formation Trajectory Planning Under Asynchronous Communication Delays

In this paper, a delay-informed multi-agent twin delayed deep deterministic policy gradient (DI-MATD3) approach is proposed for multi-UAV formation trajectory planning subject to communication delays. Specifically, a timestamped per-link communication model is first developed, in which each receiver maintains the latest valid message per link and calculates its corresponding AoI. Then, a link-level Kalman predictor propagates the cached states to the current decision instant, where the propagation step count is governed by the computed AoI. The compensated teammate states, together with their AoI values, are then integrated into the local observation space of each follower, while retaining the original actor–critic framework of MATD3. The method is evaluated in a dual-offset narrow-gate scenario involving formation compression, sequential turns, coordinated traversal, and formation recovery. Across three independent training seeds at a maximum delay of 0.6 s, Kalman+AoI achieved a task success rate of 97.33% ± 1.89%, compared with 88.67% ± 8.96% for CV + AoI and 65.33% ± 29.41% for None+AoI. Prediction-error analysis further shows that AoI-driven state reconstruction substantially reduces stale-state errors, while the errors increase during stronger maneuvers. Overall, the results support improved task-completion robustness and formation maintenance under the tested asynchronous-delay conditions, without indicating uniform superiority across all safety-related metrics.