Boosting Neural Language Inference via Cascaded Interactive Reasoning

TL;DR

Natural Language Inference (NLI) is challenged by diverse phrasing and contextual nuances. The authors introduce the Cascaded Interactive Reasoning Network (CIRN), which cascades across multiple Transformer depths to model high-order semantic interactions by forming layer-wise interaction tensors $I^{(l)} \in \mathbb{R}^{n\times m\times d}$, stacking them into $I_{\text{stack}} \in \mathbb{R}^{n\times m\times d\times L}$, and applying a DenseNet-based extractor to produce a rich feature vector for classification. They validate CIRN on ten public benchmarks, showing consistent improvements over strong PLMs and through ablations confirm the benefits of multi-layer interactions and dense-feature extraction. The approach demonstrates improved robustness and higher-order semantic modeling for NLI, with practical implications for cross-domain reasoning and paraphrase-related tasks.

Abstract

Natural Language Inference (NLI) focuses on ascertaining the logical relationship (entailment, contradiction, or neutral) between a given premise and hypothesis. This task presents significant challenges due to inherent linguistic features such as diverse phrasing, semantic complexity, and contextual nuances. While Pre-trained Language Models (PLMs) built upon the Transformer architecture have yielded substantial advancements in NLI, prevailing methods predominantly utilize representations from the terminal layer. This reliance on final-layer outputs may overlook valuable information encoded in intermediate layers, potentially limiting the capacity to model intricate semantic interactions effectively. Addressing this gap, we introduce the Cascaded Interactive Reasoning Network (CIRN), a novel architecture designed for deeper semantic comprehension in NLI. CIRN implements a hierarchical feature extraction strategy across multiple network depths, operating within an interactive space where cross-sentence information is continuously integrated. This mechanism aims to mimic a process of progressive reasoning, transitioning from surface-level feature matching to uncovering more profound logical and semantic connections between the premise and hypothesis. By systematically mining latent semantic relationships at various representational levels, CIRN facilitates a more thorough understanding of the input pair. Comprehensive evaluations conducted on several standard NLI benchmark datasets reveal consistent performance gains achieved by CIRN over competitive baseline approaches, demonstrating the efficacy of leveraging multi-level interactive features for complex relational reasoning.

Figures (1)

• Figure 1: The overall architecture of the Cascaded Interactive Reasoning Network (CIRN): 1) PLM Encoding: concatenates sentence pair ($S_1$, $S_2$) as input to a pre-trained language model (e.g., BERT, RoBERTa), obtaining representations $\mathbf{H}^{(l)}$ from each Transformer layer; 2) Multi-layer Representation: separates $\mathbf{H}^{(l)}$ into sentence representations $\mathbf{H}_1^{(l)}$, $\mathbf{H}_2^{(l)}$, and computes interaction matrices $\mathbf{M}^{(l)} \in \mathbb{R}^{n \times m \times d}$ by element-wise multiplication; 3) Feature Extraction: stacks interaction matrices across layers into an interaction tensor $\overline{\mathbf{M}} \in \mathbb{R}^{n \times m \times d \times L}$, and applies DenseNet to extract deep interactive features; 4) Prediction Layer: aggregates extracted features through a fully connected layer followed by softmax to output classification probabilities.