AI Augmented Student Advising: A Comparative Study of LLMs and Advisor-Based Course Recommendations

Course placement in higher education is pivotal for student progression. Traditional advising can be effectively augmented by Large Language Models (LLMs) to ensure scalable guidance. Crucially, we propose a novel on-premises infrastructure that runs LLMs locally, ensuring strict student data privacy. We benchmarked four open-source LLMs against gold-standard responses from human advisors (n = 3) using 35 Information Systems student profiles, identifying GPT-OSS-120B as the optimal model. To guarantee real-world viability, we profiled the infrastructure's resource utilization across 112 sequentially processed cases. Additionally, we evaluated complex scenarios: undergraduate course substitutions (n = 10) and graduate prerequisite assessments (n = 10). Results indicate strong performance in substitutions but mixed outcomes for prerequisite assessments. We also determined the optimal model parameters for recommendation accuracy. To assess broader institutional utility, we expanded testing to include four additional disciplines: Marketing, Economics, Accounting, and Finance (n = 20 per major), demonstrating the system's potential for cross-disciplinary application. This study demonstrates that securely deployed, locally hosted LLMs can provide a privacy-compliant framework to assist academic advisors, though current performance highlights the necessity of continuous human oversight.