Smart Video Analytics Solution to Identify Urban Floodborne Objects

Floating urban objects (e.g., drums, tyres, and gas cylinders) during floods is a common phenomenon that exacerbates flooding outcomes in many ways such as causing injuries, blocking infrastructure, and diverting the flows. These objects pose significant environmental and public health risks, complicate clean-up efforts, increase costs, and disrupt emergency operations. Additionally, they create disease vector breeding grounds and contribute to psychological stress among residents. Despite the critical need to identify and track floodborne urban objects, no automated approach exists. Traditionally, information on debris and damage is collected manually by emergency services, which is resource-intensive and unsafe during severe weather events. In collaboration with AVCON Projects Pty, Ltd., this research addresses the need for an automated real-time monitoring system using computer vision technologies. The goal is to develop and deploy an artificial intelligence of things system capable of processing video feeds, performing detection inference using edge-computing, tracking unique objects, and displaying counting statistics with detection bounding boxes via the real-time streaming protocol. In this context, a custom floodborne urban objects in water (FUOW) dataset has been curated from multiple sources including AVCON historical records, online databases, and controlled field experiments. A number of object detection models including the faster region-based convolutional neural network, You Only Look Once Version 4 (YOLOv4), YOLOv8, and detection transformer with improved denoising anchor boxes (DINO) were trained using FUOW, and performance was compared. Among the trained models, DINO was able to achieve the best mean average precision of 85.3%. The end-to-end solution based on the optimized trained object detection model was deployed on NVIDIA Jetson ORIN AGX using DeepStream application.