Extracting metadata from grey literature using small fine-tuned language models

The increasing volume of grey literature, such as reports, theses, presentations and blog posts produced by public sector organisations and academia, poses significant cataloguing, discoverability, and accessibility challenges in digital libraries. To help address these challenges, the National Library of Finland has been developing tools and methods for extracting metadata from grey literature publications. We have compiled a data set of Finnish grey literature that can be used for training and evaluation. Our current main approach is the use of fine-tuned small language models that can be hosted on local computing infrastructure. We have also extended the Meteor metadata extraction tool with support for LLM-based extraction. Our evaluations show that metadata extraction using our methods can achieve F1 scores of 90%–92% averaged across 13 different metadata fields. The system is ready for demonstrations, but significant work remains before it can be put into production use.