Answerability-Aware Retrieval-Augmented Generation for Enterprise Helpdesk Question Answering: An Empirical Study of Similarity Thresholds and Grounding Strategies

Retrieval-augmented generation (RAG) systems can improve question answering by grounding large language model responses in external documents. However, enterprise deployments require not only relevant answers but also reliable abstention when the available knowledge base lacks sufficient evidence. This study investigates answerability control in an enterprise helpdesk RAG system by evaluating retrieval similarity thresholds, grounding instructions, and an explicit evidence-sufficiency mechanism. Experiments used a synthetic enterprise corpus containing 23 documents and 51 retrievable chunks. Five RAG configurations were compared on a 30-question development benchmark, and the selected configuration was evaluated on a separate 50-question test benchmark. The selected configuration achieved answerability precision, recall, and F1 scores of 1.00 on the test set, with no observed false refusals or unsupported answers. Retrieval recall at k was 0.9667, while automated keyword answer accuracy was 0.90. The findings indicate that similarity-threshold selection substantially influenced false refusals in the evaluated setting, while stronger grounding instructions improved keyword-match performance. However, the small synthetic corpus and benchmark limit the generalisability of these findings.