Exploring Generative AI to Enhance Data Artifact Use in Accreditation

This study explores the viability of generative artificial intelligence (AI) tools to evaluate institutional documentation for alignment with accreditation standards. Using a portion of an institution’s evidence inventory, researchers compared manual review with AI-supported methods, including a proprietary beta tool and a publicly available large language model (LLM). Employing a deductive framework informed by Constant Comparative Analysis (CCA), as well as a quantitative comparison, the study assessed AI’s ability to place evidence in the correct standards and criteria required by the accreditor. Initial tests across 17 criteria, nested within three standards, revealed limited accuracy in category identification by LLMs. However, when the task was simplified to a broader set of standards, performance improved significantly, demonstrating LLM utility as a preliminary sorting aid. While easily accessible LLMs lack the precision and scalability required for full automation, findings point to promising directions for future development of tailored AI solutions to support accreditation.