Lightweight multi-class abnormal behavior detection algorithm in complex scenarios

Public safety monitoring faces significant challenges including dense occlusion, extreme scale variations, and environmental interference, necessitating efficient and accurate abnormal behavior detection methods. Existing deep learning models struggle to achieve an optimal balance between detection accuracy and computational efficiency, particularly in handling multi-scale targets on resource-constrained edge devices. This article proposes DOTXB-YOLO, a lightweight You Only Look Once (YOLO)-based model for multi-class abnormal behavior detection. The model incorporates four key innovations: the C3K2 GHKC module (Cross Stage Partial with kernel size 2 and Generalized Hybrid Kernel Convolution), which enhances multi-scale feature extraction capability by integrating group convolutions with 3 × 3 and 5 × 5 kernels; the Detect DLSW detection head (Decoupled Lightweight Shared-Weight), which significantly reduces parameter count through a cross-scale weight-sharing mechanism; a dual-stream downsampling strategy that effectively preserves critical spatial structural information; and TZELoss (Target Zone Enhancement Loss), which dynamically focuses on hard examples to optimize the training process. Experimental results on a custom anomaly dataset demonstrate that DOTXB-YOLO achieves 1.2% higher mean Average Precision at 50% Intersection over Union (IoU) (mAP50) than YOLO11-N while reducing both parameter count and computational complexity by approximately 22% and 23%, respectively. Cross-dataset evaluation on WiderPerson and generalization experiments on PASCAL Visual Object Classes (VOC) 2012 validate the model’s superior robustness and generalization capability, providing a practical solution for edge device deployment in public safety scenarios.