Combining feedback enhancement with knowledge graphs in large language models

Abstract Large language models (LLMs) for reasoning generation rely on their own already acquired knowledge. However, knowledge in real tasks is updated in real time, and frequent fine-tuning can be cumbersome. Recent years have witnessed the success of large-scale knowledge graphs, which could serve as an ideal domain knowledge resource. However, these large-scale knowledge graphs cannot be directly applied to LLM reasoning tasks. Moreover, they often struggle to combine the reasoning capabilities of LLM with large-scale knowledge graphs. Existing methods can hardly make use of them because the black-box structure of LLMs, which is difficult to handle. To address these problems, we present a method combining feedback enhancement with knowledge graphs in LLMs, namely FKGLM, which can automatically mining logical rules from knowledge graphs to create a domain knowledge base (KB). Meanwhile, the KB is used to perform minimum inconsistency reasoning on the initial results generated by the LLM, correcting errors in the results and updating the prompt to achieve efficient knowledge augmentation of the LLM. Experiments on three different domain tasks show that FKGLM can effectively integrate LLMs and large-scale knowledge graphs, leading to a significant enhancement in the reasoning capabilities of LLMs.