Pothole Detection and Avoidance via Deep Learning on Edge Devices

Hitting potholes can not only cause damage to vehicles but also put passengers into danger. However, due to the various factors, such as budget consideration and soil erosion, it is impossible to fix all the potholes on the roads. Instead of avoiding potholes directly, most researches tend to collect the position of potholes on the roads for developing more effective road maintenance strategies. In this paper, a pothole avoidance system deployed on an edge platform is proposed, which can avoid potholes automatically. We use Deep Q-Network (DQN) as an agent, and train it on CARLA driving simulator with cross-task unsupervised transfer learning. To accelerate the executing speed of the system on the edge platform, the model is quantized to an 8bit integer. Finally, we implement the pothole avoidance system on the Xilinx ZCU104 evaluation board, which is satisfied with the deployment limitations of vehicles, such as power consumption. To evaluate the proposed system, we have set up a virtual road environment with CARLA, and generated several potholes on the road randomly. Then, a vehicle controlled by the pothole avoidance system was tested in the environment. The evaluation results show that the proposed system can effectively reduce the opportunity of hitting potholes by 31% compared to the baseline. Besides, the system deployed on the Xilinx ZCU104 can execute at 30 FPS, and the energy efficiency is 0.91 FPS/W. In conclusion, the proposed system can effectively avoid potholes on the road, and its execution speed and power consumption meet the requirements of the actual driving situation.