SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising
Kuan Xu, Yongbo Wang, Yongliang Wang, Zujie Wen, Yang Dong
TL;DR
This paper addresses the suboptimal performance of vanilla sequence-to-sequence models on text-to-SQL by introducing SeaD, a schema-aware denoising framework that augments training with erosion and shuffle noises to explicitly model schema structure. It also proposes a clause-sensitive execution-guided decoding strategy to mitigate grammar and execution errors during inference. Through a Transformer-based architecture with a Hybrid Pointer Generator, SeaD achieves state-of-the-art results on WikiSQL, even without additional table content or schema-link annotations. The findings suggest that the capacity of standard S2S models for text-to-SQL has been underestimated when guided by task-focused denoising and decoding strategies.
Abstract
In text-to-SQL task, seq-to-seq models often lead to sub-optimal performance due to limitations in their architecture. In this paper, we present a simple yet effective approach that adapts transformer-based seq-to-seq model to robust text-to-SQL generation. Instead of inducing constraint to decoder or reformat the task as slot-filling, we propose to train seq-to-seq model with Schema aware Denoising (SeaD), which consists of two denoising objectives that train model to either recover input or predict output from two novel erosion and shuffle noises. These denoising objectives acts as the auxiliary tasks for better modeling the structural data in S2S generation. In addition, we improve and propose a clause-sensitive execution guided (EG) decoding strategy to overcome the limitation of EG decoding for generative model. The experiments show that the proposed method improves the performance of seq-to-seq model in both schema linking and grammar correctness and establishes new state-of-the-art on WikiSQL benchmark. The results indicate that the capacity of vanilla seq-to-seq architecture for text-to-SQL may have been under-estimated.