Enhancing Digital Governance Through AI Text Revision: A Comparative Evaluation Using Controlled Error Injection

This study presents a comparative evaluation of 36 Large Language Model (LLM) configurations for automating Plain Language revision in digital public service. We introduce a Controlled Error Injection methodology, in which an auxiliary agent deliberately corrupts well-formed texts while preserving their factual content. The models were then tasked with restoring clarity using a Retrieval-Augmented Generation framework based on 50 institutional guidelines. Performance was evaluated across semantic fidelity (BERTScore and Cosine Similarity), readability gains (Flesch Reading Ease index), and diagnostic explainability. The results reveal a Simplification Paradox, with models exhibiting conservative, aggressive, or equilibrium revision profiles. OpenAI and xAI models showed pronounced “black-box” behavior, lacking diagnostic transparency. In contrast, Mistral-based architectures achieved a superior balance, improving readability while preserving factual integrity and leading in explainability. Funding: This work received financial support from Finep through the INSPIRE project. Research manuscript associated with poster 028 at the III Workshop do BI0S.