Robust Pothole Avoidance in Autonomous Driving Using Multi-Task Transformer Perception and Ensemble Reinforcement Learning (2026-06-12)

Autonomous Vehicles (AVs) lack considerable reliability in unstructured contexts with road surface deterioration, particularly potholes. Existing solutions are often divided into two categories: pure perception tasks that require downstream control actionable integration, and end-to-end Reinforcement Learning (RL), which suffers from sample inefficiencies and safety hazards due to the “limited explainability” nature of neural rules. To address this gap, this paper introduces a new Hybrid Perception-Decision Framework that tightly integrates uncertainty-aware vision with an enhanced ensemble control approach. A Transformer-U-Net (TransUNet) architecture capable of estimating pothole segmentation, depth, and perceptual uncertainty all at the same time is presented. These outputs are then transformed into a compact 117-dimensional risk vector, which serves as basis of state space for a “Dual-layer Ensemble” RL controller comprising PPO, A2C, TRPO, and Recurrent PPO agents. Crucially, a deterministic Safety Refinement Layer (SRL) that serves as a kinematic guardrail, superseding RL ideas based on a Confidence Shield and Neighbour Mini-Planner is designed. A comprehensive evaluation on a unified test set of 1,180 frames indicates that the system achieves an average lateral deviation of 0.289m, reflecting a 26.3% increase in trajectory efficiency over baseline RL approaches. While RL agents demonstrated efficiency during inference, the Safety Layer interfered in 55.9% of test sequences to avoid risky manoeuvres in high-uncertainty zones, while preserving computational efficiency acceptable for near-real-time operations.