SSMOT: a self-supervised multi object detection and tracking for indoor farm animals

Wearable sensors and cameras with advanced imaging technologies play a pivotal role in monitoring animal behavior for selective breeding, managing pedigree information, and tracking genetic traits. The integration of such technological advancements is instrumental for both individual farmers and large-scale industries seeking to enhance their breeding programs. This paper proposes a camera-based multiple-animal tracking, employing a tracking-by-detection methodology within the context of self-supervised learning. In essence, the proposed framework integrates an EfficientDet detector with D0-Backbone and undergoes a two-step training process. Initially, pre-training is conducted with unlabeled data employing three self-learning strategies: Barlow Twins, SimCLR (Contrastive learning), and Masked Auto Encoder (MAE). Subsequently, the model is fine-tuned using our custom-labeled dataset in the second step. The detection results are used as inputs to our tracker, which incorporates visual and spatial information to enhance the track-to-detect association mechanism. To evaluate the effectiveness of our framework, we conducted training and testing on a proprietary dataset obtained and labeled by an animal farm in Norway. We employed standard performance metrics, including commonly used tracking measures such as Multiple Object Tracking Accuracy (MOTA), Higher Order Tracking Accuracy (HOTA), Identification Metrics (IDF1), number of ID switches (IDSW), and number of track fragments (FRAG). The quantitative results indicate that the proposed framework enhances performance on HOTA, MOTA, and IDF1 by over 8.6% on average compared to state-of-the-art methods.