Quality-by-Design Framework for Evaluating AI-Driven Machine Translation in Regulatory Affairs

Background/Objectives: Machine translation increasingly prepares multilingual product information for EU marketing authorisation, yet no established process exists to adapt or validate a system for the Summary of Product Characteristics (SmPC), whose sections differ in regulatory function, terminology, and clinical risk. This work develops such a process and a qualification procedure against the approved human translation. Methods: An EMA-derived ontology divides the SmPC into seven regulatory domains, each with a provisional ICH Q9 risk profile, following Quality-by-Design principles. A No Language Left Behind model was fine-tuned with Low-Rank Adaptation on an eight-language-pair corpus and compared with the pre-trained model using BLEU, chrF, TER and COMET. Quality was validated by a five-layer, human-sign-off pipeline. Results: Fine-tuning improved every metric for every language pair. Quality varied more by domain than by language, hidden by corpus-level scores, and the domains rated highest-risk, under provisional ratings awaiting external calibration, were not the best translated. Checks on language-independent quantities were reliable everywhere; within adverse-reaction sections, the terminology check was the only partial-anchor instrument, and it showed meaningful gains only after a second language-model stage. Conclusions: The fine-tuned model and its validation pipeline, targeting automated checks where reliable and human review elsewhere, form a promising assistive step under qualified sign-off whose suitability for production-scale deployment requires further validation; assessment must be by regulatory domain, not the document as a whole.