Fusing LIDAR, camera and semantic information: A context-based approach for pedestrian detection

In this work, a context-based multisensor system, applied to pedestrian detection in urban environments, is presented. The proposed system comprises three main processing modules: (i) a LIDAR-based module acting as the primary object detector, (ii) a module which supplies the system with contextual information obtained from a semantic map of the roads, and (iii) an image-based detection module, using sliding window detectors, with the role of validating the presence of pedestrians in the regions of interest generated by the LIDAR module. A Bayesian strategy is used to combine information from sensors onboard the vehicle (‘local’ information) with information contained in a digital map of the roads (‘global’ information). To support experimental analysis, a multisensor dataset, named the Laser and Image Pedestrian Detection dataset (LIPD), is used. The LIPD dataset was collected in an urban environment, under daylight conditions, using an electrical vehicle driven at low speed. A down-sampling method, using support vectors extracted from multiple linear SVMs, was used to reduce the cardinality of the training set and, as a consequence, to decrease the CPU time during the training process of the image-based classifiers. The performance of the system is evaluated, in terms of detection rate and the number of false positives per frame, using three image-detectors: a linear SVM, a SVM-cascade, and a benchmark method. Additionally, experiments are performed to assess the impact of contextual information on the performance of the detection system.