Operational Use and AI Governance of LLM-Assisted Split Procurement Detection in University Internal Auditing

This study examines how large language model (LLM) assessments were used in an actual university internal audit. We analyzed system outputs, audit records, and accounts from two meetings with audit staff. A cloud implementation of a previously developed method processed 1,860,622 voucher line items covering 12 months. Candidate analysis covered eight organizational units. Across three monetary thresholds, 949 threshold-specific candidates were assigned to the three higher review-priority categories. After audit-specific filtering, auditors reviewed approximately 700 candidates, exported 36 for detailed examination, and selected four for departmental inquiry. They compared their own assessments with LLM-generated rationales and retained some uncertain candidates for further investigation. They also used priorities to decide how thoroughly to review candidates, depending on the volume in each department. Thus, LLM output informed both the interpretation of transactions and the allocation of review effort. These findings extend research on human oversight by showing how scrutiny of LLM rationales can coexist with reliance on priorities when allocating review effort. Independent evaluation of detection performance and time savings remains necessary.