What question did this study set out to answer?

The aim is to improve execution accuracy of small open-source models for text-to-SQL tasks by decomposing the problem into simpler components.

January 1, 2024Open Access

DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models

Key Points

The aim is to improve execution accuracy of small open-source models for text-to-SQL tasks by decomposing the problem into simpler components.
Proposed a two-stage fine-tuning approach for text-to-SQL tasks.
Evaluated the method on three large cross-domain datasets and two small language models.
Assessed the performance against proprietary large language models.
Achieved 60.31% execution accuracy on the BIRD hold-out test set with a 7B parameter model.
Improved execution accuracy by 3 to 7 percent compared to previous methods.
Aligned performance of open-source models closer to proprietary counterparts.

Abstract

Leading models for the text-to-SQL task heavily rely on proprietary Large Language Models (LLMs), posing concerns over data privacy.Closing the performance gap between small open-source models and large proprietary models is crucial to mitigate this reliance.To this end, we introduce a novel two-stage fine-tuning approach that decomposes the task into two simpler tasks.Through comprehensive evaluation on three large cross-domain datasets and two small LLMs, we show that this approach improves execution accuracy by 3 to 7 percent, effectively aligning the performance of open-source models with their proprietary counterparts.Our proposed method has achieved 60.31% execution accuracy on BIRD hold-out test set, which is the highest performance among methods using 7B parameter models.

اسأل الذكاء الاصطناعي

Bookmark

View Full Paper