Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Atomic-SNLI: Fine-Grained Natural Language Inference through Atomic Fact Decomposition

Minghui Huang

TL;DR

This paper addresses the interpretability gap in sentence-level natural language inference by introducing atomic-level NLI through atomic fact decomposition. It presents Atomic-SNLI, a large-scale dataset built by decomposing SNLI into atomic facts and generating high-quality atomic-level entailment, neutral, and contradiction examples. Fine-tuning models on Atomic-SNLI yields substantial improvements in atomic-level reasoning, especially for hypotheses with multiple atomic facts, while preserving strong sentence-level accuracy. The work also demonstrates how probabilistic aggregation over atomic predictions enables transparent, robust, and explainable inferences, establishing a new benchmark and methodology for fine-grained, compositional NLI. Overall, Atomic-SNLI enables more granular, interpretable reasoning and points to future directions in more sophisticated aggregation and broader applicability in textual understanding tasks.

Abstract

Current Natural Language Inference (NLI) systems primarily operate at the sentence level, providing black-box decisions that lack explanatory power. While atomic-level NLI offers a promising alternative by decomposing hypotheses into individual facts, we demonstrate that the conventional assumption that a hypothesis is entailed only when all its atomic facts are entailed fails in practice due to models' poor performance on fine-grained reasoning. Our analysis reveals that existing models perform substantially worse on atomic level inference compared to sentence level tasks. To address this limitation, we introduce Atomic-SNLI, a novel dataset constructed by decomposing SNLI and enriching it with carefully curated atomic level examples through linguistically informed generation strategies. Experimental results demonstrate that models fine-tuned on Atomic-SNLI achieve significant improvements in atomic reasoning capabilities while maintaining strong sentence level performance, enabling both accurate judgements and transparent, explainable results at the fact level.

Atomic-SNLI: Fine-Grained Natural Language Inference through Atomic Fact Decomposition

TL;DR

This paper addresses the interpretability gap in sentence-level natural language inference by introducing atomic-level NLI through atomic fact decomposition. It presents Atomic-SNLI, a large-scale dataset built by decomposing SNLI into atomic facts and generating high-quality atomic-level entailment, neutral, and contradiction examples. Fine-tuning models on Atomic-SNLI yields substantial improvements in atomic-level reasoning, especially for hypotheses with multiple atomic facts, while preserving strong sentence-level accuracy. The work also demonstrates how probabilistic aggregation over atomic predictions enables transparent, robust, and explainable inferences, establishing a new benchmark and methodology for fine-grained, compositional NLI. Overall, Atomic-SNLI enables more granular, interpretable reasoning and points to future directions in more sophisticated aggregation and broader applicability in textual understanding tasks.

Abstract

Current Natural Language Inference (NLI) systems primarily operate at the sentence level, providing black-box decisions that lack explanatory power. While atomic-level NLI offers a promising alternative by decomposing hypotheses into individual facts, we demonstrate that the conventional assumption that a hypothesis is entailed only when all its atomic facts are entailed fails in practice due to models' poor performance on fine-grained reasoning. Our analysis reveals that existing models perform substantially worse on atomic level inference compared to sentence level tasks. To address this limitation, we introduce Atomic-SNLI, a novel dataset constructed by decomposing SNLI and enriching it with carefully curated atomic level examples through linguistically informed generation strategies. Experimental results demonstrate that models fine-tuned on Atomic-SNLI achieve significant improvements in atomic reasoning capabilities while maintaining strong sentence level performance, enabling both accurate judgements and transparent, explainable results at the fact level.
Paper Structure (46 sections, 4 figures, 8 tables)

This paper contains 46 sections, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fact Decomposition in NLP.
  4. Decomposition Methods.
  5. Fine-grained NLI.
  6. Atomic-Level NLI Analysis
  7. Hypothesis Decomposition
  8. Inference Framework
  9. Inference Rules.
  10. Experimental Setup
  11. Results and Analysis
  12. Model Biases and Limitations.
  13. Atomic-SNLI Dataset Construction
  14. Data Construction Methodology
  15. Hypothesis Decomposition.
  16. ...and 31 more sections

Figures (4)

  • Figure 1: How Sentence-Level and Atomic-Level NLI Explains Reasoning. Top (Sentence-Level): The model reads the full premise and hypothesis and gives one answer: "contradiction." The reason for this decision is unclear. Bottom (Atomic-Level): The hypothesis is broken into single facts. The model checks each fact against the premise. These individual checks are combined to reach the final "contradiction" verdict, clearly showing which part caused the conflict.
  • Figure 2: Performance metrics (Accuracy, Precision, Recall) of nli-deberta-v3-base across different numbers of atomic facts per hypothesis, categorized by NLI labels.
  • Figure 3: Atomic-SNLI construction pipeline: Original SNLI pairs are decomposed into atomic facts, then processed through three label-specific pipelines to generate high-quality entailment, contradiction, and neutral pairs. $\tau_{rel} \in \{\tau_e, \tau_n, \tau_c\}$.
  • Figure 4: The prompt used to generate contradiction pairs for atomic facts. The model is instructed to produce a minimal edit that creates a direct and unambiguous contradiction with the premise.