Towards a Transitional Weather Scene Recognition Approach for Autonomous Vehicles

Driving in adverse weather conditions is a key challenge for autonomous vehicles (AV). Typical scene perception models perform poorly in rainy, foggy, snowy, and cloudy conditions. In addition, we observe transition states between extremes (cloudy to rainy, rainy to sunny, etc.) in nature with variations in adversity. It is crucial to define and understand these transition states in order to develop robust AV perception models. Existing research works on classification focused on identifying extreme weather conditions. However, there is a lack of emphasis on the transition between these extreme weather scenes. Hence, this paper proposes an approach to define and understand six intermediate weather transition states: sunny to rainy, rainy to sunny, and others. Firstly, we propose a way to interpolate the intermediate weather transition data using a variational autoencoder and extract its spatial features using VGG. Further, we model the temporal distribution of these spatial features using a gated recurrent unit to classify the corresponding transition state. Also, we introduce a large-scale dataset called the AIWD6: Adverse Intermediate Weather Driving dataset, generated for three different time intervals. Experimental results on the AIWD6 dataset demonstrate that our model efficiently generates weather transition conditions for AV technology. Also, the spatio-temporal deep neural network can effectively classify the adverse weather transition states for different time intervals.