Deploying Large Language Models on edge platforms with mobile-oriented resource constraints faces challenges of limited resources and hallucination issues. While Retrieval-Augmented Generation (RAG) mitigates hallucinations through external knowledge, existing RAG systems on such platforms suffer from poor retrieval quality and lack standardized protocols. We propose a RAG framework for edge platforms with mobile-oriented resource constraints based on the Model Context Protocol (MCP), enabling plug-and-play access to heterogeneous knowledge bases. Our framework introduces a weighted voting fusion ranking mechanism integrating BERT-Recall, F1, Relaxed Exact Match (REM), and Query Relevance scores to enhance retrieval accuracy, combined with model quantization and few-shot learning for efficient on-device operation. Experiments on SQuAD, HotpotQA, and TriviaQA demonstrate that our framework achieves higher accuracy and lower error rates than state-of-the-art methods while maintaining low latency.
Zhang et al. (Tue,) studied this question.