Bridging the Semantic Gap: Contrastive Rewards for Multilingual Text-to-SQL with GRPO

TL;DR

This work tackles the semantic alignment challenge in multilingual Text-to-SQL by integrating a multilingual contrastive reward with Generalized Reward Policy Optimization (GRPO). A contrastive encoder based on XLM-RoBERTa-grounded embeddings provides a continuous semantic signal that guides SQL generation toward user intent, beyond mere execution correctness. Using a 3B Llama model with LoRA fine-tuning on the MultiSpider dataset, the approach achieves strong execution and semantic accuracy, notably surpassing zero-shot baselines and approaching larger models while using only 3,000 training examples. The results highlight the practical significance of semantically-aware reinforcement learning for cross-lingual semantic parsing and demonstrate notable gains across seven languages, including Vietnamese, with a data-efficient, scalable training recipe.

Abstract

Current Text-to-SQL methods are evaluated and only focused on executable queries, overlooking the semantic alignment challenge -- both in terms of the semantic meaning of the query and the correctness of the execution results. Even execution accuracy itself shows significant drops when moving from English to other languages, with an average decline of 6 percentage points across non-English languages. We address these challenges by presenting a new framework that combines Group Relative Policy Optimization (GRPO) within a multilingual contrastive reward signal to enhance both task efficiency and semantic accuracy in Text-to-SQL systems in cross-lingual scenarios. Our method teaches models to obtain better correspondence between SQL generation and user intent by combining a reward signal based on semantic similarity. On the seven-language MultiSpider dataset, fine-tuning the LLaMA-3-3B model with GRPO improved the execution accuracy up to 87.4 percent (+26 pp over zero-shot) and semantic accuracy up to 52.29 percent (+32.86 pp). Adding our contrastive reward signal in the GRPO framework further improved the average semantic accuracy to 59.14 percent (+6.85 pp, up to +10 pp for Vietnamese). Our experiments showcase that a smaller, parameter-efficient 3B LLaMA model fine-tuned with our contrastive reward signal outperforms a much larger zero-shot 8B LLaMA model, with an uplift of 7.43 pp in execution accuracy (from 81.43 percent on the 8B model to 88.86 percent on the 3B model), and nearly matches its semantic accuracy (59.14 percent vs. 68.57 percent) -- all using just 3,000 reinforcement learning training examples. These results demonstrate how we can improve the performance of Text-to-SQL systems with contrastive rewards for directed semantic alignment, without requiring large-scale training datasets.

Figures (5)

• Figure 1: End-to-End Training and Evaluation Pipeline. The process begins with the MultiSpider dataset with parallel NL questions in 7 languages and corresponding SQL. A multilingual contrastive encoder is trained to align semantically equivalent questions across languages. This encoder provides a contrastive semantic reward signal during policy fine-tuning. The Generalized Reward Policy Optimization (GRPO) fine-tuning uses: (1) an execution reward ($R_{\text{exec}}$) giving a binary signal if the SQL yields correct results, and (2) a contrastive reward ($R_{\text{contrastive}}$) based on cosine similarity between question embeddings. These rewards update the LLM policy to produce SQL that is both executable and semantically faithful to the user's intent.
• Figure 2: Comparison of Execution Reward vs. Semantic (Contrastive) Reward. While execution rewards only verify if SQL queries run correctly on a database (binary signal), our contrastive rewards provide nuanced feedback about semantic alignment between user intent and generated SQL across languages, capturing subtle differences that execution signals miss. This is particularly important for multilingual queries where translation can introduce semantic ambiguities that binary execution signals cannot address.
• Figure 3: Execution Accuracy (top) and Semantic Accuracy (bottom) across different models and languages. Our L3B-GRPO-C model (dark blue) consistently outperforms the baseline L3B model and approaches or exceeds the larger L8B model, particularly in Execution Accuracy.
• Figure 4: Example query where contrastive reward improves semantic accuracy while maintaining execution accuracy. The model without contrastive rewards uses $\geq$ instead of $>$ as specified in the question and doesn't count distinct movies.
• Figure 5: Visual comparison of semantic accuracy with and without contrastive rewards across languages. Green annotations show the percentage point improvements achieved with our contrastive reward approach.