DCMM-SQL: Automated Data-Centric Pipeline and Multi-Model Collaboration Training for Text-to-SQL Model

TL;DR

DCMM-SQL introduces a fully automated data-centric pipeline for text-to-SQL that targets data quality and leverages multi-model collaboration. It combines adaptive data repair, error data augmentation, and a two-step MM training regime with an ensemble-based final selection to boost accuracy on lightweight models. The approach demonstrates strong performance on Bird and Spider benchmarks, achieving top results among lightweight configurations and highlighting the value of data-centric strategies in conjunction with diversified model training. This framework offers practical pathways to improve real-world text-to-SQL systems without relying solely on scaling up model size.

Abstract

Text-to-SQL tasks have gained attractive improvements since the release of ChatGPT. Among them, agent-based frameworks have been widely used in this field. However, the impact of data-centric strategies on text-to-SQL tasks has rarely been explored. In this paper, we systemically design a fully automated data-centric pipeline for text-to-SQL tasks, including \emph{adaptive data repair}, which can automatically find and fix errors in the training dataset; and \emph{error data augmentation}, where we specifically diffuse and enhance erroneous data predicted by the initially trained models. Meanwhile, we propose a Multi-Model collaboration training schema, aiming to train multiple models with different augmented data, enabling them to possess distinct capabilities and work together to complement each other, because it has been found that the capability of a single fine-tuned model is very limited. Furthermore, we utilize an ensemble strategy to integrate the capabilities of multiple models to solve a multiple-choice question, aiming to further improve the accuracy of text-to-SQL tasks. The experiment results and ablation study have demonstrated the effectiveness of data-centric pipeline and Multi-Model(MM) interactive iterative strategies, achieving first place in lightweight text-to-SQL models (within 70B).

Figures (5)

• Figure 1: The overall structure of the DCMM-SQL method.
• Figure 2: The overall structure of our propsoed Multi-LLMs collaboration training consists of two steps: first, 1) preliminary training with original data, then 4) active learning training with augmented erroneous data, where erroneous training data is found and repaired in 2) adaptive data repair, and then diffused and augmented by 3) error data augmentation. ($Q$: question, $DS$: database schema, $SQL$: original annotated SQL, $SQL'$: predicted SQL by init text-to-SQL model)
• Figure 3: The overall structure of the data synthesis, augmentation, and verification method.
• Figure 4: Data-centric pipeline analysis results.
• Figure 5: Some examples of incorrect annotations in the Bird training set.