Investigating Approaches to Automated Subject Indexing – From Lexical Matching and Supervised XMLC to Generative (L)LMs

Algorithms for automated subject indexing have seen enormous dynamics in recent years. While the open-source toolkit Annif has established a widely accepted framework for integrating automated subject indexing services into library processes, the underlying algorithms are subject to change. While old-school lexical matching may still be a valid choice, specialised algorithms for supervised Extreme Multi Label Classification (XMLC) have dominated benchmarks for a decade and the next generation of automated subject indexing solutions announces itself to be powered by generative AI. In this talk we will compare these competing and complementing approaches to automated subject indexing and we will present results of a benchmark study recently conducted at the German National Library.