Decoding virtual chats: NLP insights into academic library services.

Assessing unstructured data from virtual reference chats is complex. Full-text reveals nuances but is time-consuming, while transcript metadata gives an overview but may miss important details in the conversation. This research applies a machine learning (ML) tool to the complete set of transcripts from a research university's chat reference service (2017–2022) to examine evolving trends and patron needs in the library reference service. The study has two key objectives: 1) demonstrating ML's effectiveness in the academic library setting, and 2) assessing the impact of COVID-19 on chat reference needs. A text classification model, trained on 1.5 % of the sample, achieves a 75 % accuracy match with human annotations. Findings indicate a marked rise in circulation-related inquiries as libraries transitioned to fully online services during the pandemic. Notably, user behaviors remain consistent even after the pandemic. This study highlights ML's potential to analyze large-scale unstructured data effectively in the academic library setting. • Method for analyzing virtual reference transcripts uses ML-based multiclass classification to categorize text into topics. • Prodigy, a command-line tool built on spaCy, was used for transcript annotation and model training. • The model achieved 75 % accuracy with only 1.5 % of the data annotated, verified against human-labeled benchmarks. • 88,000 virtual reference transcripts were analyzed, providing insights in VRS needs distribution and traffic change. • No significant shifts in library VRS usage patterns despite operation disruption from the pandemic.