Technical report demonstrates transparent, community-led FAIR assessments across digital research repositories, highlighting reproducible evaluation methods for research software engineers.
RSECon 2026 Event Type: Talk Abstract While the FAIR Principles are widely endorsed, assessment remains a "black box" of subjective, tool-specific tests that often do not return the same score for the same digital object (https://fairassist.org/tools). For RSEs, this lack of standardisation hinders effective evaluation and use of these tools, which would otherwise be helpful in ensuring FAIR compliance at every stage, from initial planning to preservation, sharing, and reuse. We introduce an OSTrails framework supporting reusable, transparent, and modifiable FAIR assessments. It will also cover various tools implementing this framework, including: the FAIRassist registry, a Vue3 web-tool for inspecting Dimensions (high-level principles), Metrics (community-specific interpretations of those principles), and Benchmarks (logical groupings of metrics); a FAIR-Test endpoint, which uses the FAIR Champion tool to execute tests stored within a public FAIRsharing GitHub repository, and a framework-compliant dashboard developed with CESSDA. We will also showcase a new institutional repository FAIR benchmark co-developed by FAIRsharing and the Oxford Research Archive. Supported by the UKRN FIRM Project, this benchmark has been reviewed by nine UK and three European universities, reflecting a demand for transparent, institutional-level evaluation. RSEs will learn how to integrate these "FAIR-by-design" workflows into their own environments, adopting a transparent, reusable method of defining FAIR that is increasingly being incorporated into the UK and European research landscapes. Prerequisites This session is designed for RSEs from all disciplines and backgrounds. While no specific technical prerequisites are required, attendees will benefit from a general interest in research data quality and a foundational understanding of the FAIR Principles. This session will link high-level policy requirements around FAIR with technical implementation, making it relevant for anyone involved in planning, sharing, or preserving research outputs. Outcomes Attendees will gain a clear understanding of how to move (and why they should!) from opaque, tool-specific FAIR scoring to transparent, community-governed assessments. By exploring the OSTrails framework and the FAIRassist registry, RSEs will be equipped to: Learn how the FAIR principles can be interpreted for different communities and places within the research data life cycle. Implement "FAIR-by-design" benchmarks that ensure data quality at any stage of the research lifecycle using open-source tools and APIs. Utilize reusable metrics, tests and benchmarks, such as the new UKRN-supported institutional repository benchmark, to provide consistent evaluation across different projects. Acknowledgements The authors would like to acknowledge the Oxford Research Archive (ORA), with whom the initial draft of the institutional repository benchmark was written and the functional implementation created. We also thank the contributing members (Cardiff University, the University of Glasgow, and the University of Sheffield) and the five observing members (Keele University, Newcastle University, the University of Bristol, the University of Leicester, and the University of Wolverhampton) of the UK Reproducibility Network (UKRN) Open Research Programme, who are reviewing this benchmark within the FIRM project. Special thanks go to John Shepherdson (CESSDA) for the FAIR dashboard and our ongoing collaboration as part of OSTrails. This work has received support from the UKRN, and the authors have received funding in OSTrails from the European Union’s Horizon Europe framework programme under grant agreement No. 101130187.
No takes yet. Share an insight, caveat, or question.
Lister et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: