Real-time livestock monitoring system comprising cattle vocalization detection and individual identification

This study aims to develop an automated system for monitoring livestock by detecting and analyzing cattle vocalizations in real-time. As livestock farming expands, technologies are essential for efficient cattle management and health assessment. We focus on cattle vocalizations as a non-invasive, behaviorally meaningful signal of internal states. Cattle vocalize in response to discomfort or abnormal conditions, making calls important welfare indicators. Detecting and transmitting these vocalizations to farm managers enables early intervention and continuous, contact-free monitoring. To realize this, we propose an integrated system that performs real-time vocalization segmentation, individual identification, and cattle state detection. For segmentation, we use the WhisperSeg model, based on the Whisper Transformer pre-trained for Automatic Speech Recognition (ASR), applying transfer learning from human Voice Activity Detection (VAD) to animal VAD and fine-tuning it with cattle vocalizations. Identification and state detection are conducted using Support Vector Machines (SVM) and features from intermediate layers of the AVES (Animal Vocalization Encoder based on Self-Supervision) model, capturing cattle-specific vocal patterns. We compare layers to identify those best representing vocal traits. To handle unknown individuals, we implement Open-set Recognition and optimize the “unknown” threshold using evaluation data. This system supports real-time livestock monitoring, improving welfare and efficiency in large-scale farms.