Hybrid motion-based object detection for detecting and tracking of small and fast moving drones

A hybrid detecting and tracking system that is made especially for small and fast moving drones in which the other existing techniques cannot detect is presented. This new automatic system integrates geometry estimation, 2D object detection, 3D localization, trajectory estimation and tracking for dynamic scene interpretation. This caters to moving cameras, e.g. those mounted on a separate flying platform. The technique is built on the unified two-step real-time algorithm for detection and tracking of moving objects (DATMO) in dynamic outdoor environments. The first step is built on the motion-based detection approach for early detection of an object when the targeted object is still far away and appears very small in the scene. The second step is based on the appearance-based detection approach for continual detection, further identification and verification of the object. The camera's own position is estimated by 3D triangulation of the static landscape in the scene, doubled up as ground truth. This includes both the pan-tilt-zoom motion and the camera's vibration compensation. Therefore, the drone can be tracked, even when the camera is in motion and/or under jittery condition. As a result, the position of the tracked drone in the global inertial coordinate is further improved. Subsequently, the drone motion is tracked using the Extended Kalman Filter scheme. Initial results show that this hybrid system is able to detect a small drone from a distance away in less than a few frames after such a drone appears in the scene. The drone is always tracked as long as it is in the camera's field of view. As this technique is vision-based, this drone detection and tracking system can expand to detect multiple drones.