BHL-MOR: Background Assisted Heatmap Learning for Moving Object Recognition

Moving object recognition (MOR), which aims to detect and classify moving objects in video sequences, is a critical task in intelligent video analysis. Existing MOR methods face two fundamental challenges: detection-by-segmentation paradigms suffer from error propagation and fragmented masks, while spatial-temporal approaches rely on constrained sliding windows that limit temporal coverage for background estimation. To address these issues, we propose a novel Background-assisted Heatmap Learning framework for Moving Object Recognition (BHL-MOR). Our framework introduces a learning-free, temporally-extended background estimation paradigm that maintains a cache of historical background samples using minimum-weight replacement, enabling long-term scene recovery without explicit foreground segmentation. Unlike conventional approaches that cascade background subtraction with static object detection, we design a unified keypoint-based detector that takes both the current frame and the estimated background as inputs. This architecture incorporates task-aligned coordinate attention and motion-conditioned heatmap learning to explicitly discriminate moving objects from static distractors, directly predicting center heatmaps and bounding boxes. Extensive experiments on the challenging CDNET-MOR dataset demonstrate that BHL-MOR achieves 87.1% mAP50, outperforming prior MOR-specific methods, keypoint detectors, and recent transformer-based models, while reducing computational overhead by 3.6 times compared to MotionRecV1.