A Text-to-SQL strategy based on large language models and knowledge graphs for real-world databases

The Text-to-SQL task is defined as “given a relational database and a natural language sentence that describes a question on the database, generate an SQL query over the database that expresses the question”. Text-to-SQL strategies based on Large Language Models (LLMs) achieve remarkable performance on well-known benchmarks, but their performance is significantly less for real-world databases. Some of the reasons for this performance decrease lie in the mismatch between the end user’s view of the data and the organization and naming conventions of the database schema, and in the differences between the end user’s data semantics and the encoding of such semantics in the database. This article then proposes an LLM-based strategy to compile natural language questions into SQL queries that uses a knowledge graph and incorporates a dynamic few-shot examples technique. The implementation of the strategy leverages a database keyword search tool and specific naming conventions to expose the knowledge graph to an LLM thereby simplifying the text-to-SQL task. The article includes experiments with a real-world, proprietary relational database and the Mondial database to assess the performance of the proposed strategy. The experiments suggest that the strategy achieves an accuracy on challenging relational databases that surpasses state-of-the-art approaches on the same databases.