PromoCraft+: a human-in-the-loop framework for auditing AI-generated insurance promotions

Purpose This paper examines how generative AI is reshaping financial-services marketing by focusing on the information quality of AI-generated insurance SMS messages. It addresses three gaps: how recipients evaluate relevance and soundness, how these perceptions relate to quantitative text metrics, and how human oversight can be structured to reduce communication risk in a regulated context through the PromoCraft+ framework. Design/methodology/approach A mixed-methods design was used. A total of 450 insurance promotional SMS messages were generated using three LLMs under zero-shot, one-shot, and few-shot conditions. Quantitative evaluation was conducted at the model level using readability, formality, concreteness, and ROUGE, with ANOVA used to test differences across models. Qualitative evidence was collected through 19 semi-structured interviews and analysed using the Gioia methodology, guided by Eppler's relevance and soundness dimensions. Findings The results show significant model-level differences in linguistic quality across readability, formality, concreteness, and ROUGE. However, the qualitative findings show that recipient judgments depend on more than linguistic fluency. Messages were valued when they were clear, concise, and internally consistent, but were judged less favourably when they lacked context, appeared exaggerated, or did not feel personally useful. The findings therefore indicate only partial convergence between machine-evaluated metrics and human judgments, supporting the need for structured human oversight in regulated marketing communication. Research limitations/implications The findings are based on insurance SMS and may not generalize directly to other industries, channels, or longer-form content. Practical implications The study offers a replicable framework for assessing AI-generated marketing content in high-trust, compliance-sensitive settings. Social implications The framework supports more ethical, transparent, and audience-sensitive deployment of AI in marketing communication. Originality/value The paper introduces PromoCraft+, a human-in-the-loop audit framework that connects automated screening with recipient-level evaluation and coded interpretation. It also contributes a domain-specific INK-10 dataset and extends Eppler's Information Quality Model into the context of AI-generated insurance promotions.