Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsing

TL;DR

This work tackles zero-resource cross-lingual semantic parsing by introducing Cross-lingual Back-Parsing (CBP), a data-augmentation framework that synthesizes target-language utterances from source meaning representations using a multilingual seq2seq backbone with language adapters. A novel source-switched denoising objective trains adapters to control output language, enabling generation of $u_{tgt}$ from $mr_{src}$, followed by a round-trip consistency filter to ensure semantic fidelity. Empirical results on Mschema2QA and Xspider show substantial improvements in target-language exact-match and high slot-value alignment, even with only monolingual data and no target-language labels, outperforming several MT-based baselines and competitive LLM prompts. The findings demonstrate practical, scalable zero-resource cross-lingual SP and suggest CBP's applicability to other cross-lingual generation tasks.

Abstract

Recent efforts have aimed to utilize multilingual pretrained language models (mPLMs) to extend semantic parsing (SP) across multiple languages without requiring extensive annotations. However, achieving zero-shot cross-lingual transfer for SP remains challenging, leading to a performance gap between source and target languages. In this study, we propose Cross-Lingual Back-Parsing (CBP), a novel data augmentation methodology designed to enhance cross-lingual transfer for SP. Leveraging the representation geometry of the mPLMs, CBP synthesizes target language utterances from source meaning representations. Our methodology effectively performs cross-lingual data augmentation in challenging zero-resource settings, by utilizing only labeled data in the source language and monolingual corpora. Extensive experiments on two cross-language SP benchmarks (Mschema2QA and Xspider) demonstrate that CBP brings substantial gains in the target language. Further analysis of the synthesized utterances shows that our method successfully generates target language utterances with high slot value alignment rates while preserving semantic integrity. Our codes and data are publicly available at https://github.com/deokhk/CBP.

Figures (10)

• Figure 1: An overview of the data augmentation process with CBP. The utterance generator equipped with the language adapter $A_{t}$ synthesizes utterances in the target language $t$, and a filtering mechanism is applied to discard low-quality utterances. In the example, a Korean utterance is generated, with the corresponding English translation provided in parentheses.
• Figure 2: Overview of the utterance generator training. First, we individually train language adapters, represented by colored boxes in the decoder, using monolingual corpora for each language through source-switched denoising training. The remaining shared parameters are frozen during this process (\ref{['fig:step1']}). Next, we use labeled data to fine-tune the utterance generator for the utterance generation task, keeping the trained adapters frozen while selectively training the other parameters (\ref{['fig:step2']}). The figure is adapted from ustun2021multilingual.
• Figure 3: During source-switched denoising training, the language identity switch operation $\Phi$ switches the language identity of the encoded representation of the masked sentence $\clubsuit$ from language $l$ to the source language, resulting in $\spadesuit$.
• Figure 4: Target language synthesis rates (from 0 to 1) on different languages in Mschema2QA. For more granular results, refer to Appendix Table \ref{['tab:syn_dist']}.
• Figure 5: Quality of synthesized utterances measured by GEMBA-stars. We use gpt-3.5-turbo as the backbone.