Disentangling the Past: Relation and Event Extraction with Schema-Constrained LLM Generation

Extracting relational and event data from historical printed sources at scale is a central but unsolved measurement problem in political science. This paper introduces an open-source Python package (text2rel) that turns unstructured printed sources into network-ready data-pairing layout-preserving OCR, evidence-grounded retrieval, and schema-constrained large language model (LLM) outputs-with an evaluation framework that makes the resulting measurement error visible and auditable. Rather than answering isolated questions as in question-answering (QA) systems, the pipeline extracts many relations and events jointly under a fixed schema, links every record to its source passage, and routes passages to human coders, benchmarking LLM ground-truth agreement against inter-and intra-coder validation. Evaluated against a handlabeled sample from a 52-volume Colombian genealogical corpus, both tasks are recall-limited, with recall exceeding precision (relations ≈ 0.79/0.72; events ≈ 0.76/0.70 at the θ = 0.80 cut). Relation scores rise faster as the threshold relaxes-as expected, since a triplet has fewer components than a quintuple and a higher chance of a full-tuple match. Error is argument-limited : the relation type and event verb are almost always recovered (≈ 0.99, ≈ 0.96), so residual error sits in argument fields-names, dates, places-not the categorical core. Two diagnostics prove informative though imperfect: match quality declines with input length, and LLM self-reported confidence weakly tracks correctness. The package enables using these signals in an iterative workflow, letting users refine prompts against assessed evidence. As a face-validity check, the reconstructed elite kinship network recovers sensible structure: elected presidential candidates are more betweenness-central and denser in their ego-networks than unsuccessful ones. A future version will characterize hallucinations directly and benchmark cloud against local open-weights models.