Enhancing Effectiveness and Robustness in a Low-Resource Regime via Decision-Boundary-aware Data Augmentation

TL;DR

The paper tackles robustness in low-resource NLP by introducing decision-boundary-aware data augmentation that shifts latent representations toward the decision boundary and reconstructs ambiguous sentences with soft labels, aided by mid-$K$ sampling for diversity. The approach combines a pretrained language model with a gradient-based augmentation step, guided by a trained attribute classifier, and is augmented by a decoding strategy that preserves semantics while expanding variation. Ablation studies show the value of soft labels and curriculum augmentation, and empirical results demonstrate improved accuracy and stronger adversarial robustness across multiple datasets and models. This method reduces reliance on external LLMs while enhancing robustness and diversity of training data, offering a practical path for improving low-resource NLP systems.

Abstract

Efforts to leverage deep learning models in low-resource regimes have led to numerous augmentation studies. However, the direct application of methods such as mixup and cutout to text data, is limited due to their discrete characteristics. While methods using pretrained language models have exhibited efficiency, they require additional considerations for robustness. Inspired by recent studies on decision boundaries, this paper proposes a decision-boundary-aware data augmentation strategy to enhance robustness using pretrained language models. The proposed technique first focuses on shifting the latent features closer to the decision boundary, followed by reconstruction to generate an ambiguous version with a soft label. Additionally, mid-K sampling is suggested to enhance the diversity of the generated sentences. This paper demonstrates the performance of the proposed augmentation strategy compared to other methods through extensive experiments. Furthermore, the ablation study reveals the effect of soft labels and mid-K sampling and the extensibility of the method with curriculum data augmentation.

Figures (2)

• Figure 1: The figure illustrates the concept of the decision-boundary-aware gradient modification. In the previous method, augmentation was performed without the consideration of decision boundaries. However, in the proposed method, augmentation is performed in decision-boundary-aware manner.
• Figure 2: The figure illustrates the concept of the mid-K Sampling. Mid-K Sampling is a method to increase the diversity of generated sentences by sampling the middle K sentences instead of selecting the K sentences with the highest probability values (Top-K Sampling).