Geodesic Active Contour Based Fusion of Visible and Infrared Video for Persistent Object Tracking

Persistent object tracking in complex and adverse environments can be improved by fusing information from multiple sensors and sources. We present a new moving object detection and tracking system that robustly fuses infrared and visible video within a level set framework. We also introduce the concept of the flux tensor as a generalization of the 3D structure tensor for fast and reliable motion detection without eigen-decomposition. The infrared flux tensor provides a coarse segmentation that is less sensitive to illumination variations and shadows. The Beltrami color metric tensor is used to define a color edge stopping function that is fused with the infrared edge stopping function based on the grayscale structure tensor. The min fusion operator combines salient contours in either the visible or infrared video and drives the evolution of the multispectral geodesic active contour to refine the coarse initial flux tensor motion blobs. Multiple objects are tracked using correspondence graphs and a cluster trajectory analysis module that resolves incorrect merge events caused by under-segmentation of neighboring objects or partial and full occlusions. Long-term trajectories for object clusters are estimated using Kalman filtering and watershed segmentation. We have tested the persistent object tracking system for surveillance applications and demonstrate that fusion of visible and infrared video leads to significant improvements for occlusion handling and disambiguating clustered groups of objects