Exploring Large Language Models for Text-to-SQL Error Correction with LoRA Fine-Tuning

This is the authors' abstract. We don't add key points for this paper.

The advent of Large Language Models (LLMs) has significantly advanced Text-to-SQL tasks in recent years. However, these models still face considerable challenges when applied in practice, such as hallucinations inherent in LLMs and the rigid syntactic structure of SQL, which often lead to the generation of erroneous SQL queries. Existing approaches typically rely on older models or simplistic LLM prompts for direct error correction, which is a relatively basic strategy. In this paper, we explore the performance of large models in the Text-to-SQL error correction task, focusing on the combination of LoRA fine-tuning and retrieval-augmented methods. Experimental results demonstrate that our approach achieves state-of-the-art (SOTA) performance on the Splash benchmark, outperforming existing methods and providing more accurate error corrections for SQL generation. Notably, despite using a 7B-parameter model, our method surpasses the performance of much larger 72B-scale models, highlighting the efficiency and effectiveness of our approach.