Visual Surveillance of Human Activities via Gradient-Based Coverage Control on Matrix Manifolds

In this article, we address visual surveillance of human activities for a network of cameras with controllable orientations based on gradient-based coverage control techniques. We first formulate the problem as an optimization problem on the matrix manifold SO(3) and then derive the gradient for the cost function using a density function defined on the image plane of each camera. We then develop a real-time density estimation algorithm using computer vision techniques including a real-time pedestrian detection algorithm and examine its real-time feasibility through simulation. We finally demonstrate the complete algorithm including the gradient descent algorithm, the density estimation algorithm, image acquisition/processing and physical motion on a simulator built on a 3-D animation software and an experimental testbed.