Fusion-Based Utilization and Synthesis of Efficient Detections

This study aimed to develop hardware and software for an object detection fusion system, using three different sensors. The system was built and studied with the motivating application of autonomous drones searching for and detecting people in a search-and-rescue scenario. The system’s performance was compared to that of individual sensors deployed for the same task. The focus of the research was to prove the competence and benefits of a decision-level fusion method as it was applied to a lightweight object detection architecture, and the driving motivators behind the study were simplicity in implementation and good computational performance. In short, the algorithm would use lightweight models to perform object detection on the sensors individually, and then the detection coordinates would be transformed into the same coordinate planes for correlation. The method exploited sensor calibration and depth images to create an efficient workflow. Once the algorithm was created, it was tested and compared both qualitatively and quantitatively to individual models deployed on the same sensors with no fusion framework. Qualitative results clearly indicated that the fusion algorithm outperformed individual models in more challenging scenarios. Quantitative results were generated by computing average precision for the workflow and the individual models, and the workflow retained a performance of 90% even when introduced to challenging scenarios. Meanwhile, two of the individual models degraded to below 80% and 70% performance. These results indicated the same trend as the qualitative results, but it was also clear that inaccuracies in the fusion methodology resulted in a small percentage of the true detections being missed when they were otherwise caught by individual models. Future work should consider investigating the deployment of the fusion algorithm on small devices, because the lightweight models were intended for mobile deployments. Other possible work should study the effect of improved extrinsic calibration, better-trained models, semantic segmentation models overlayed for improved depth resolution, and adding adaptability to the algorithm’s decision-making process for different scenarios.