Learning patterns of human activity for anomaly detection

Commercial security and surveillance systems offer advanced sensors, optics, and display capabilities but lack intelligent processing. This necessitates human operators who must closely monitor video for situational awareness and threat assessment. For instance, urban environments are typically in a state of constant activity, which generates numerous visual cues, each of which must be examined so that potential security breaches do not go unnoticed. We are building a prototype system called BALDUR (Behavior Adaptive Learning during Urban Reconnaissance) that learns probabilistic models of activity for a given site using online and unsupervised training techniques. Once a camera system is set up, no operator intervention is required for the system to begin learning patterns of activity. Anomalies corresponding to unusual or suspicious behavior are automatically detected in real time. All moving object tracks (pedestrians, vehicles, etc.) are efficiently stored in a relational database for use in training. The database is also well suited for answering human- initiated queries. An example of such a query is, "Display all pedestrians who approached the door of the building between the hours of 9:00pm and 11:00pm." This forensic analysis tool complements the system's real-time situational awareness capabilities. Several large datasets have been collected for the evaluation of the system, including one database containing an entire month of activity from a commercial parking lot.