This study presents an AI-based, human-in-the-loop decision support system designed for large-scale institutional query routing and response generation. The proposed system combines semantic text classification with large language model-based response generation to assist administrative staff in handling high-volume natural language requests from various system users, while preserving human oversight. Using a dataset of 135,359 real student and staff interactions collected over 15 years, the system was designed, deployed, and evaluated in a live university information portal. The classification component achieved 95.88% accuracy in evaluation and 82.21% staff acceptance in practice, while 94.81% of AI-generated draft responses were adopted with minor edits. Operational evaluation showed a 30.8% reduction in resolution time, a 32.6% decrease in misrouting, and an increase in user satisfaction from 3.6 to 4.9 out of 5. The system is implemented as a modular RESTful API to ensure interoperability with existing Student Information Systems, with analysis code available upon request to support replication in similar resource-constrained environments. The results illustrate how human-in-the-loop AI systems can support improvements in service quality, efficiency, and institutional capacity in resource-constrained environments, providing a transferable applied AI framework for scalable decision support in complex administrative domains.
Eyupoglu et al. (Wed,) studied this question.