Automated FHWA Vehicle Classification System via LiDAR and Camera Data Fusion

The FHWA classification scheme provides a standardized method for vehicle classification based on the number of axles and axle configuration, making it essential for effective traffic management, toll collection, and transportation planning. Although image-based classification methods have advanced significantly, they often struggle to accurately distinguish between certain FHWA vehicle classes using visual data alone. This challenge has led researchers to combine classes that are difficult to differentiate visually, thereby reducing the granularity and effectiveness of classification systems. In this research, we present a novel approach that enhances vehicle classification by leveraging the complementary strengths of both camera and LiDAR data. Because of the lack of existing fine-grained camera-LiDAR data sets for vehicle classification, we collected our own comprehensive dataset for the FHWA 13 classes. We employ a pretrained YOLOv8 model to detect and localize vehicles. Features from these localized images are extracted using a ResNet model. Simultaneously, the localized LiDAR point clouds are used to generate distance maps and calculate vehicle features such as length and height. The combined features and dimensions are input into a camera-LiDAR fusion multilayer perceptron classifier for final classification. To evaluate the performance of our proposed models, we employed metrics including recall, precision, and F1 score. Our comparative analysis demonstrates that the model fusing images and the distance maps yielded the highest performance, achieving a precision of 0.958, a recall of 0.932, and an F1 score of 0.938 across all classes.