Evaluation of Multiclass Traffic Sign Detection and Classification Methods for U.S. Roadway Asset Inventory Management

Frequent analysis and updating the condition of traffic signs and mile markers are among the most important aspects of a highway asset management system. Today’s practices mainly involve manual data collection and analysis, which have to be done for millions of miles of roads and the practice needs to be repeated regularly. While significant progress has been made on improving the data collection practice by leveraging video streams collected from car-mounted cameras, the analysis has primarily remained a manual and labor-intensive process. Automating the analysis from the collected videos is also challenging due to the interclass variability of traffic signs, expected changes in illumination, occlusion, sign position, and orientation. To address these challenges, this paper presents and evaluates the performance of three computer vision algorithms for detection and classification of traffic signs in presence of cluttered backgrounds and static and dynamic occlusions. The task particularly focuses on (1) extracting two-dimensional (2D) candidate windows from already collected video streams that potentially contain traffic signs–without making any prior assumption about their locations; (2) detecting the presence of signs in these 2D candidate windows; and (3) classifying them into warning, regulatory, stop, and yield sign categories based on their shape and color. For validation, a new comprehensive benchmark data set of over 11,000 annotated U.S. traffic sign images with a large range of pose, scale, background, illumination, and occlusion variation is introduced. Experimental results show an average accuracy of 76.20%, 89.31%, and 94.83% for the methods of (1) Haar-like features with Cascade classifiers, (2) histograms of oriented gradients (HOG) with multiple one-versus-all support vector machine (SVM) classifiers, and (3) HOG+C with the SVM classifiers, a variant of the second method with histograms of colors concatenated to HOG. The experimental results demonstrate the potential of leveraging joint representation of texture and color in HOG+C together with SVM discriminative classifiers as a viable solution for creating up-to-date and complete inventories of traffic signs for U.S. roadways.