Promise and pitfalls: Variable performance of AI-assisted passive acoustic monitoring of birds and frogs using BirdNET and VicFrogNET

Effective conservation relies on accurate data on species distributions and their change over time. Recent advances in low-cost acoustic sensors and artificial intelligence (AI) have transformed ecoacoustics into a scalable tool for automated, large-scale wildlife monitoring. However, two of the major limitations current AI models face are: (1) tools like BirdNET are widely used despite limited understanding of what drives variation in detection accuracy across species; and (2) equivalent open-access models for non-avian taxa such as amphibians are largely absent. We address both gaps by presenting the first open-source, multi-species frog classifier for 16 native species from southeastern Australia (VicFrogNET). To evaluate its performance, we used more than 1.5 TB of real-world passive acoustic data collected from 60 freshwater wetlands spanning approximately 300 km and analysed these recordings with both VicFrogNET and the default BirdNET bird classifier. This dataset contained 820,740 bird detections (172 species) and 2419 frog detections (15 species), with >17,000 detections manually validated. Both classifiers exhibited a bimodal performance pattern, with species tending to be detected either with very high (>90%) or very low (<25%) precision. Misclassification was influenced by call similarity, background noise, and vocal complexity. Notably, VicFrogNET, despite being trained on a modest dataset, produced consistently low false-positive rates across most species. This suggests that training data quality may be more important than quantity for achieving reliable performance in targeted regional classifiers, demonstrating the potential of data-efficient models for local applications. We release our frog classifier, dataset, and documentation as open-access resources to promote transparency, reproducibility, and uptake. These tools enable rapid screening of large audio volumes, allowing users to prioritise target species detections and accelerate biodiversity assessments at scale.