From video to behaviour: An LSTM ‐based approach for automated nest behaviour recognition in the wild

Abstract Studies of animal behaviour usually rely on direct observations or manual annotations of video recordings. However, such methods can be very time‐consuming and error‐prone, leading to sub‐optimal sample sizes. Recent advances in deep learning show great potential to overcome such limitations. Nevertheless, most currently available behavioural recognition solutions remain focused on captivity settings. Here, we present a deployment‐focused framework to guide researchers in building behavioural recognition systems from video data, using Long Short‐Term Memory (LSTM) networks to classify behavioural sequences across consecutive frames. LSTMs allowed us to: (1) monitor nest activity by detecting the birds' presence and simultaneously classifying the type of trajectory: i.e. nest‐chamber entrance or exit; and (2) identify the behaviour performed: building, aggression or sanitation. Our framework achieved comparable error rates to human annotators while greatly outperforming them in speed. Model performance improved with challenging training instances and remained robust even with modest sample sizes. LSTM also outperformed YOLO (‘You Only Look Once’), highlighting the critical role of temporal sequence information in behavioural analysis. We demonstrate that our approach is replicable across three bird species and applicable to deployment videos, highlighting its value as a generalisable and transferable tool for long‐term studies in the wild.