Rationalization Models for Text-to-SQL
Gaetano Rossiello, Nhan Pham, Michael Glass, Junkyu Lee, Dharmashankar Subramanian
TL;DR
Rationalization Models for Text-to-SQL addresses the challenge of producing interpretable, stepwise reasoning for text-to-SQL by generating Chain-of-Thought rationales. The authors combine a small set of manual annotations with dynamic few-shot knowledge distillation from a teacher LLM to train a rationalization model that can produce abundant synthetic CoTs, enabling full training coverage. The framework improves execution accuracy on the BIRD benchmark, particularly for moderate and challenging queries, while providing explainable query-building processes. This approach has practical implications for enterprise text-to-SQL use, enabling scalable CoT annotation and cross-domain applicability.
Abstract
We introduce a framework for generating Chain-of-Thought (CoT) rationales to enhance text-to-SQL model fine-tuning. These rationales consist of intermediate SQL statements and explanations, serving as incremental steps toward constructing the final SQL query. The process begins with manually annotating a small set of examples, which are then used to prompt a large language model in an iterative, dynamic few-shot knowledge distillation procedure from a teacher model. A rationalization model is subsequently trained on the validated decomposed queries, enabling extensive synthetic CoT annotations for text-to-SQL datasets. To evaluate the approach, we fine-tune small language models with and without these rationales on the BIRD dataset. Results indicate that step-by-step query generation improves execution accuracy, especially for moderately and highly complex queries, while also enhancing explainability.