Answers are grounded in this paper and cite their sources.
Surveillance in controlled environments such as military bases and other restricted facilities requires timely detection of suspicious activities. This paper presents a real-time multi-mode deep learning framework that integrates YOLOv8-based person detection, Deep SORT multi-object tracking, spatio-temporal feature extraction, and behavior recognition through rule-based, supervised, and self-learning operating modes. A six-mode ablation study compares hybrid and no-fallback variants under a shared deployment pipeline. Tracking evaluation on the PETS09 surveillance dataset confirms stable person trajectories suitable for downstream analysis, and a two-stage sensitivity study identifies operational thresholds for loitering duration, radius, and running speed. Under independent manual ground-truth validation on a controlled deployment video, the self-learning hybrid mode achieved the highest accuracy and macro F1 among the six evaluated modes, while supervised-synthetic and rule-based modes remain competitive. Removing the rule-based fallback causes both LSTM-only ablations to degrade toward near-random behavior, demonstrating that fallback fusion is functionally essential. Self-learning achieves this performance without any manually labeled deployment data, suggesting potential practical applicability for controlled surveillance environments, pending further validation. Preliminary cross-scene evaluation on a second deployment video shows that all hybrid modes maintain high agreement with a rule-based reference and reproduce the same no-fallback degradation pattern across scenes. On the manually annotated deployment subset, the self-learning hybrid mode attains an accuracy of 0.899 and a macro F1 of 0.713, while both no-fallback ablations collapse to near-random accuracy of approximately 0.50, a 38-40 percentage-point accuracy drop across the two ablations relative to their fallback-enabled counterparts, indicating that rule-based fallback fusion is necessary for maintaining above-chance performance in this framework.
No takes yet. Share an insight, caveat, or question.
Mushad et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: