ABEX: Data Augmentation for Low-Resource NLU via Expanding Abstract Descriptions

TL;DR

ABEX addresses data scarcity in low-resource NLU by proposing Abstract-and-Expand, a two-stage augmentation framework. It learns to expand abstract descriptions with a BART model trained on a large synthetic dataset of abstract-document pairs and generates downstream abstractions controllably via AMR editing. The downstream augmentations are produced by converting documents to AMR abstracts, expanding them, and optionally fine-tuning on the downstream data; after the initial ABEX training, the method is largely training-free for new datasets. Evaluations across 12 datasets and 4 NLU tasks under four low-resource settings show that ABEX yields superior context, token, and length diversity and outperforms prior methods, indicating strong practical impact for data-scarce NLP.

Abstract

We present ABEX, a novel and effective generative data augmentation methodology for low-resource Natural Language Understanding (NLU) tasks. ABEX is based on ABstract-and-EXpand, a novel paradigm for generating diverse forms of an input document -- we first convert a document into its concise, abstract description and then generate new documents based on expanding the resultant abstraction. To learn the task of expanding abstract descriptions, we first train BART on a large-scale synthetic dataset with abstract-document pairs. Next, to generate abstract descriptions for a document, we propose a simple, controllable, and training-free method based on editing AMR graphs. ABEX brings the best of both worlds: by expanding from abstract representations, it preserves the original semantic properties of the documents, like style and meaning, thereby maintaining alignment with the original label and data distribution. At the same time, the fundamental process of elaborating on abstract descriptions facilitates diverse generations. We demonstrate the effectiveness of ABEX on 4 NLU tasks spanning 12 datasets and 4 low-resource settings. ABEX outperforms all our baselines qualitatively with improvements of 0.04% - 38.8%. Qualitatively, ABEX outperforms all prior methods from literature in terms of context and length diversity.

Figures (6)

• Figure 1: Illustration of our proposed augmentation methodology. Top: Learning to Expand Abstract Descriptions. ① We synthesize a large-scale synthetic dataset $\mathcal{D}_{ab}$ with abstract-document pairs by prompting LLMs with unlabeled documents from $\mathcal{D}_{ab}$. ② We pre-train BART on this dataset with abstract as input and document as the target for learning to expand abstract descriptions. Bottom: Data Augmentation. ① We convert the document into its AMR graph representation $\mathcal{G}_{i}$ using a Text-to-AMR Parser. ② $\mathcal{G}_{i}$ then goes through multiple steps of deletion to obtain $\hat{\mathcal{G}}_{i}$ ③ We optionally retrieve a semantically similar document from $\mathcal{D}_{down}$, obtain its AMR graph $\mathcal{G}_{k}$, and replace subtrees in $\hat{\mathcal{G}}_{i}$ with similar subtrees in $\hat{\mathcal{G}}_{i}$. ④ $\hat{\mathcal{G}}_{i}$ is then converted back to text (which is now an abstract description) using an AMR-to-Text generator. ⑤ This abstract description is then passed to the fine-tuned BART for generating augmentations. ⑥ We optionally fine-tune the fine-tuned BART (from the 1st step) on abstract-document pairs from $\mathcal{D}_{down}$.
• Figure 2: Comparison of augmentations on the MultiCoNER dataset (500 setting). ABEX not only introduces novel contexts of varying lengths around existing NEs but also introduces new NEs. More examples in Fig. \ref{['fig:atis']}, \ref{['fig:mrpc']}, and \ref{['fig:yahoo']}.
• Figure 3: Augmentation examples on the ATIS dataset. All generations are produced in a low-resource setting (500 training examples).
• Figure 4: Augmentation examples on the MRPC dataset. All generations are produced in a low-resource setting (500 training examples).
• Figure 5: Augmentation examples on the SQuAD dataset. All generations are produced in a low-resource setting (500 training examples).