VT-FLAC: A visible-thermal fusion network for robust wildlife monitoring in open-field environments using near-remote sensing imagery

• A visible-thermal dataset for sheep and Tibetan wild ass on Qinghai-Tibet Plateau. • VT-FLAC model employs dual-branch interaction for efficient cross-modal fusion. • Reduces target counting MAE by 5% with > 92% localization accuracy (nAP@0.5). • Shows strong generalizability in near-remote sensing biodiversity monitoring. The utilization of near-remote sensing imagery for animal population counting and localization frequently faces substantial challenges, mainly attributed to blurred or occluded targets. To evaluate animal counting and localization under such realistic conditions, we present a novel visible-infrared counting dataset consisting of 465 pairs of sheep images and 400 pairs of Tibetan wild ass images, all collected on the Qinghai-Tibet Plateau during the peak activity season (August to September) to capture diverse behavioral patterns. Based on the aforementioned practical challenges, this paper proposes a Visible-Thermal Fuzzy Localization and Animal Counting (VT-FLAC) model. The proposed VT-FLAC model builds upon a point-based regression paradigm and introduces a task-oriented dual-branch interaction framework driven by a Cross-modal Adaptive Feature Aggregation (CAFA) module. Unlike existing cross-modal attention fusion approaches that mainly perform global feature alignment, VT-FLAC explicitly integrates cross-modal attention into the point regression process, enabling adaptive spatial refinement under the geometric variability and blur commonly observed in near-ground remote sensing. To address the prevalent density imbalance in wildlife populations, we further propose a Hybrid Regression-Classification (HRC) loss that jointly optimizes localization and counting, particularly improving robustness in dense and low-saliency regions. Comparative experiments demonstrate that VT-FLAC approach outperforms ten other object counting methods on our visible-thermal dataset. For sheep, the proposed method Mean Absolute Error (MAE) by 78.38% and Mean Squared Error (MSE) by 73.43% relative to the second-best method, while achieving 92.60% nAP{0.5} in localization. For Tibetan wild asses, the proposed method lowers MAE by 5.85% and MSE by 6.62%, with localization accuracy reaching 93.80% nAP{0.5}. Moreover, comparative experiments performed on public datasets provide further evidence of the robustness and generalizability of the proposed method across diverse scenarios. It can also maintain excellent accuracy with MAE of 10.35 and MSE of 8.10 on the dataset of unseen species in a brand-new scenario, which is 49.09% and 61.94% lower than the second-best model respectively. Ablation study shows that our core contribution is a holistic fusion framework that effectively combines spatial-channel coordination with the dedicated HRC loss. HRC loss proves to be the more impactful component, alone contributing to reductions of 1.40 in MAE and 1.29 in MSE for counting on average, alongside a 3.55% gain in localization nAP{0.5}. The full integration of both elements yields the best overall robustness against blur, occlusion, and weak saliency. This study develops a cross-modal deep learning framework for accurate fuzzy animal counting and localization, thereby facilitating the advancement of near-remote sensing imagery and AI applications in biodiversity management.