A New Benchmark Dataset and Mixture-of-Experts Language Models for Adversarial Natural Language Inference in Vietnamese

TL;DR

This work presents ViANLI, the first adversarial NLI dataset for Vietnamese, generated through a human-in-the-loop process to yield >10k premise-hypothesis pairs from diverse news sources and enriched reasoning types. It also introduces NLIMoE, a Mixture-of-Experts model with dynamic routing atop a shared encoder to tackle adversarial NLI, achieving 47.3% accuracy on ViANLI and outperforming XLM-R Large. Training on ViANLI also improves performance on additional Vietnamese NLI benchmarks, indicating the dataset’s value as both a robust evaluation resource and an effective training corpus. Overall, ViANLI advances robustness testing for Vietnamese NLI, while NLIMoE demonstrates the potential of adaptive expert routing for complex linguistic inference in low-resource languages.

Abstract

Existing Vietnamese Natural Language Inference (NLI) datasets lack adversarial complexity, limiting their ability to evaluate model robustness against challenging linguistic phenomena. In this article, we address the gap in robust Vietnamese NLI resources by introducing ViANLI, the first adversarial NLI dataset for Vietnamese, and propose NLIMoE, a Mixture-of-Experts model to tackle its complexity. We construct ViANLI using an adversarial human-and-machine-in-the-loop approach with rigorous verification. NLIMoE integrates expert subnetworks with a learned dynamic routing mechanism on top of a shared transformer encoder. ViANLI comprises over 10,000 premise-hypothesis pairs and challenges state-of-the-art models, with XLM-R Large achieving only 45.5% accuracy, while NLIMoE reaches 47.3%. Training with ViANLI improves performance on other benchmark Vietnamese NLI datasets including ViNLI, VLSP2021-NLI, and VnNewsNLI. ViANLI is released for enhancing research into model robustness and enriching resources for future Vietnamese and multilingual NLI research.

Figures (14)

• Figure 1: Overall Process for the Creation of the ViANLI Dataset.
• Figure 2: Example of Word Overlap in the Premise-hypothesis Pair of the Entailment Label. Where the Green Highlight Is the Same Words, the Red Highlight Is the New Words That Only Appear in the Hypothesis Sentence.
• Figure 3: Example of POS-tag in the Premise-hypothesis Pair of the Entailment Label. The Red Words in the Hypothesis Tree Are New Words, and the Arrows Represent Changes in the Position and Function of the Word.
• Figure 4: The New Word Rate by Part-of-Speech in ViANLI.
• Figure 5: Analysis of the combination ratio of Inference Types on the ViANLI Dev Set.