Answers are grounded in this paper and cite their sources.
Implementation study demonstrates an AI-assisted workflow analyzing developer transcripts in educational program evaluation, highlighting improved accuracy without requiring domain expertise.
External evaluators of NSF Advanced Technological Education (ATE) projects are often asked to document technical progress in fields where they are not the experts. This article describes a replicable, artificial intelligence (AI)-assisted workflow that we developed during the Year 2 evaluation of KonnectXR (KXR): Extended Reality in Technician Education (NSF Award Nos. 2435348-2435353) to address that problem. The workflow applies a structured, large language model (LLM)-assisted reading of developer meeting transcripts, a source that records technical milestones and decisions as they happen. Using it, we documented the development team’s progress more accurately and relied less on our own technical expertise. The resulting evidence fed directly into NSF goal-level reporting. We close with what the approach means for evaluators of technically complex ATE projects.
No takes yet. Share an insight, caveat, or question.
Hodgson et al. (2026) studied this question.