Reproducible Workflows by Semantic Provenance Description - Finding Common Ground

We present an approach to semantically represent benchmarks and benchmark runs to verify and validate numerical algorithms and software packages basing on existing standards (RO-Crate, PROV, metadata4ing) and infrastructures (RO-Hub, NFDI4ING Jupyter Service) . Research Objects in form of RO-Crates allow to pack data together with a small local knowledge graph in form of a JSON-LD file describing the data, its provenance and related information entities in a semantic way basing on schema.org. We use RO-Crates in two different ways: To bring together all the files and information required to run a benchmark, and - basing on the Workflow Run RO-Crate profile suite - to capture the output of a benchmark together with its provenance description, both in a semantic and machine actionable way. The data model bases on schema.org, PROV-O, metadata4ing and the ontologies underlying the MathModDB and MathAlgoDB of MaRDI and the open research knowledge graph (ORKG) and takes into account the Cross-Domain Interoperability Framework (CDIF). A snakemake exporter plugin allows to automatically generate the provenance enriched RO-Crate according to the Workflow Run RO-Crate profile. The resulting RO-Crate objects can be combined in a flexible way to build and query an ad-hoc knowledge graph of benchmarks and their results. While RO-Hub is used as an open infrastructure to host the RO-Crates, the NFDI4ING Jupyter Service allows to query and visualize the results.