Using Webby FDOs to Integrate AI Taxon Identification and Citizen Science

Camera traps and passive acoustic devices are particularly useful in providing non-invasive methods to document wildlife diversity, ecology, behavior, and conservation. The application of autonomous Internet of Things (IoT) sensors is constantly developing and opens up new application possibilities for research and nature conservation such as taxon identification based on real-time audio processing in the field (Höchst et al. 2022). Furthermore, the amount of associated recorded digital photos, videos, and audio files is growing at a rapid pace, producing too much data to make human annotation feasible. Machine-learning, in contrast, can generate (baseline) annotations at scale on high-throughput data but may not capture all details compared with the complex contextual understanding of human annotators. We developed the WildLIVE platform*1 to effectively combine machine-learning (taxon and individual identification of animals) and “Human-in-the-Loop” citizen science (Fig. 1, Jansen et al. 2024). Our primary objective is to combine crowdsourced curation of digital image, audio, and video content from biodiversity monitoring with (semi-)autonomous data processing by machines, and subsequently mobilize these data as machine-actionable knowledge units, i.e., FAIR Digital Objects (FDO, Wittenburg et al. 2023) To this effect, the data model of WildLIVE features a “webby" FDO approach leveraging web-based components involving Research Object Crate (RO-Crate) and FAIR Signposting to enable packaging of an observing process’ contextual information (e.g., metadata of sensors, geolocation, and links to content stream), together with operational semantics giving machines the information needed to autonomously process (e.g., detect regions of interest in images, Younis et al. 2020) the actual data (Soiland-Reyes et al. 2024). To represent the semantics of data capture events, we designed an ontology, the WildLife Monitoring Ontology (WLMO), which provides a formal description of fundamental concepts and relations. The talk will provide an overview of the platform’s development status and the technology stack employed (combining RO-Crate and FAIR Signposting with AI plus “Humans-in-the-Loop”) for data exchange with emerging data infrastructures such as the Common European Data Spaces.