Retrieval-Augmented Large Language Model-Based Framework for Hierarchical Classification of Public Feedback on Transportation Infrastructure

Transportation agencies receive large volumes of free-form public comments describing infrastructure conditions, safety concerns, and service issues. These comments are often processed manually for downstream operational actions, which is time-consuming, inconsistent across reviewers, and difficult to scale, thereby limiting their value for operational decision-making. This study presents a machine learning and Large Language Model (LLM) framework for automated triage of free-form public comments, assigning each report to a three-level hierarchical taxonomy consisting of Category, Subcategory, and Final Decision. The proposed framework uses agency historical data together with retrieval-based evidence, where semantically similar past comments are provided to the LLM as contextual support to better align predictions with agency-specific labeling practices. The framework was evaluated using TF-IDF with Logistic Regression, TF-IDF with Linear SVM, embedding-based kNN with cosine similarity, few-shot LLM prompting, and retrieval-based LLM prompting. Results show that retrieval-based prompting achieved the best overall performance, with the highest accuracy at both the Category and Subcategory levels. At the Final Decision level, retrieval-based prompting slightly outperformed kNN, while few-shot prompting performed worse. Error analysis showed that many misclassifications were semantically plausible alternatives, reflecting the overlap across infrastructure-related complaint categories. When a second candidate label was allowed, further improving performance. Latency analysis also indicated that the framework can process more than 2000 comments in under 30 min, supporting faster and more consistent agency workflows.