Rapid fish sound detection using human-in-the-loop deep learning

Passive acoustic monitoring (PAM) can be used to detect and classify marine fauna sounds, providing a powerful, non-invasive tool for studying animal presence and behaviour across various temporal and spatial scales, offering unique insights into marine ecosystems. The use of PAM specifically for fish is currently limited by the intense manual effort to annotate the audio data. While deep learning has enabled automated PAM data processing in bird and marine mammal studies, these approaches depend on large annotated training datasets. Such datasets are not available for most fish species, especially in temperate waters where sounds are less frequent and harder to detect and identify. Therefore, this study evaluates an Agile Modelling workflow that incorporates a human-in-the-loop approach to efficiently train sound detectors with minimal effort. Previously applied to birds and reef sounds, we assess its applicability in two temperate marine environments, markedly different from prior test cases. The workflow allows for models to be trained in under two hours with no other initial training data than one example of the target sound. The detectors trained were evaluated against manually annotated datasets. Results show that the Agile Modelling workflow can effectively train models for detecting rare and putative fish sounds, significantly reducing annotation time. Different strategies were compared to offer practical guidelines and highlight method limitations. This approach enables quicker model development, promotes the sharing of annotated datasets, and could accelerate the broader adoption of automated fish PAM. Ultimately, such tools support improved monitoring, understanding and conservation of marine ecosystems.