Regulatory Churn Evaluation Framework (RCEF): A Reproducible Methodology for Evaluating Retrieval Systems Under Document Evolution

Retrieval-augmented generation systems are typically evaluated against static benchmarks that assume a fixed, unchanging corpus — an assumption that fails for enterprise and regulatory repositories, which evolve continuously through revision, supersession, expiration, and amendment. Under these conditions, a retrieval system may return semantically relevant documents that are nevertheless invalid for a query's temporal or applicability conditions, a failure mode conventional relevance-based benchmarks do not detect. This paper presents the Regulatory Churn Evaluation Framework (RCEF), a controlled, reproducible methodology for evaluating retrieval correctness under document evolution. RCEF specifies an empirically calibratable churn taxonomy; explicit algorithms for constructing versioned corpora and churn-distributed queries; a mechanical, architecture-independent admissibility labeling procedure with a multi-rater reliability requirement for real-world corpora; six admissibility-centric metrics; and a bootstrap-based statistical validation procedure. We demonstrate RCEF with a single illustrative implementation, summarizing its use under increasing document churn and repository governance degradation.