Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI

Sourav Banerjee, Anush Mahajan, Ayushi Agarwal, Eishkaran Singh

TL;DR

This paper tackles the plateau in Natural Language Inference (NLI) performance on SNLI by introducing UnitedSynT5, which couples synthetic data augmentation with the Entailment Few-Shot Learning (EFL) framework. A FLAN-T5 XL generator creates premise–hypothesis pairs from SNLI data, guided by few-shot prompts, and the outputs are cleaned and converted into EFL-formatted data before training a GTR-T5-XL classifier on the expanded dataset. The approach yields new state-of-the-art results across SNLI (94.7%), E-SNLI (94.0%), and MultiNLI (92.6%), demonstrating that synthetic data and model scaling can substantially enhance NLI generalization. These findings highlight the value of data diversity and iterative refinement in NLI systems, suggesting a viable direction for advancing natural language understanding tasks in practice.

Abstract

Natural Language Inference (NLI) tasks require identifying the relationship between sentence pairs, typically classified as entailment, contradiction, or neutrality. While the current state-of-the-art (SOTA) model, Entailment Few-Shot Learning (EFL), achieves a 93.1% accuracy on the Stanford Natural Language Inference (SNLI) dataset, further advancements are constrained by the dataset's limitations. To address this, we propose a novel approach leveraging synthetic data augmentation to enhance dataset diversity and complexity. We present UnitedSynT5, an advanced extension of EFL that leverages a T5-based generator to synthesize additional premise-hypothesis pairs, which are rigorously cleaned and integrated into the training data. These augmented examples are processed within the EFL framework, embedding labels directly into hypotheses for consistency. We train a GTR-T5-XL model on this expanded dataset, achieving a new benchmark of 94.7% accuracy on the SNLI dataset, 94.0% accuracy on the E-SNLI dataset, and 92.6% accuracy on the MultiNLI dataset, surpassing the previous SOTA models. This research demonstrates the potential of synthetic data augmentation in improving NLI models, offering a path forward for further advancements in natural language understanding tasks.

First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI

TL;DR

This paper tackles the plateau in Natural Language Inference (NLI) performance on SNLI by introducing UnitedSynT5, which couples synthetic data augmentation with the Entailment Few-Shot Learning (EFL) framework. A FLAN-T5 XL generator creates premise–hypothesis pairs from SNLI data, guided by few-shot prompts, and the outputs are cleaned and converted into EFL-formatted data before training a GTR-T5-XL classifier on the expanded dataset. The approach yields new state-of-the-art results across SNLI (94.7%), E-SNLI (94.0%), and MultiNLI (92.6%), demonstrating that synthetic data and model scaling can substantially enhance NLI generalization. These findings highlight the value of data diversity and iterative refinement in NLI systems, suggesting a viable direction for advancing natural language understanding tasks in practice.

Abstract

Natural Language Inference (NLI) tasks require identifying the relationship between sentence pairs, typically classified as entailment, contradiction, or neutrality. While the current state-of-the-art (SOTA) model, Entailment Few-Shot Learning (EFL), achieves a 93.1% accuracy on the Stanford Natural Language Inference (SNLI) dataset, further advancements are constrained by the dataset's limitations. To address this, we propose a novel approach leveraging synthetic data augmentation to enhance dataset diversity and complexity. We present UnitedSynT5, an advanced extension of EFL that leverages a T5-based generator to synthesize additional premise-hypothesis pairs, which are rigorously cleaned and integrated into the training data. These augmented examples are processed within the EFL framework, embedding labels directly into hypotheses for consistency. We train a GTR-T5-XL model on this expanded dataset, achieving a new benchmark of 94.7% accuracy on the SNLI dataset, 94.0% accuracy on the E-SNLI dataset, and 92.6% accuracy on the MultiNLI dataset, surpassing the previous SOTA models. This research demonstrates the potential of synthetic data augmentation in improving NLI models, offering a path forward for further advancements in natural language understanding tasks.

Paper Structure

This paper contains 20 sections, 3 equations, 1 figure, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Natural Language Inference
  4. Entailment Few-Shot Learning (EFL)
  5. Methodology: Overcoming the Current Performance Limitations
  6. Generation of Synthetic Dataset
  7. Training the FLAN-T5 XL Generator to Generate Hypotheses
  8. Prompt Design and Structure
  9. Iterative Process for Generator Training
  10. Output and Dataset Augmentation
  11. Dataset Cleaning
  12. Filtering Criteria
  13. Dataset Reduction
  14. EFL Conversion
  15. Training a New GTR-T5-XL Model
  16. ...and 5 more sections

Figures (1)

  • Figure 1: Overview of the proposed approach for enhancing Natural Language Inference (NLI) performance through synthetic data augmentation.