Answers are grounded in this paper and cite their sources.
Benchmarking study evaluates ESCOX skill extraction accuracy in European job vacancies, highlighting areas where automated parsing diverges from expert annotations.
Labour markets across Europe increasingly describe workers through skills rather than job titles, and a growing number of large language model (LLM)-based tools now claim to extract these skills automatically from unstructured text at scale. Among these, ESCOX has gained particular traction for its open-source, taxonomy-aligned design, yet like any LLM-based system it remains susceptible to hallucination, prompt sensitivity, and non-deterministic output, risks that are rarely quantified before such tools are deployed in practice. The European Skills, Competences, Qualifications, and Occupations (ESCO) classification provides the standardised reference against which this risk can be measured, but no study has yet benchmarked an ESCO-aligned LLM extractor against an independent, expert-labelled dataset at scale. This study addresses that gap. Candidate skills generated by ESCOX are compared against reference skills already assigned to job vacancies on the EURES portal by national labour-market experts, using job-by-skill matrices to quantify agreement and skill co-occurrence networks to characterise how the two sets diverge structurally. Results reveal the extent to which ESCOX’s automatic output aligns with expert judgement and where systematic divergences occur. These findings offer HR practitioners, policymakers, and labour-market researchers an evidence-based basis for deciding when ESCOX’s output can be trusted directly and when expert oversight remains necessary.
No takes yet. Share an insight, caveat, or question.
Kavargyris et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: