Shortcut-Stacked Sentence Encoders for Multi-Domain Inference

TL;DR

The paper tackles multi-domain natural language inference (Multi-NLI) with an encoding-based approach. It introduces Shortcut-Stacked Sentence Encoders: a stack of biLSTM layers with shortcut connections and word embedding fine-tuning, producing fixed-length sentence vectors fed into a three-way entailment classifier via feature matching. Across Multi-NLI matched and mismatched settings and SNLI, the approach achieves strong improvements over prior encoders and sets state-of-the-art encoding-based results, including top single-model performance in RepEval 2017. An Addendum shows that residual connections can reduce parameters while maintaining comparable accuracy, highlighting a trade-off between shortcut and residual designs.

Abstract

We present a simple sequential sentence encoder for multi-domain natural language inference. Our encoder is based on stacked bidirectional LSTM-RNNs with shortcut connections and fine-tuning of word embeddings. The overall supervised model uses the above encoder to encode two input sentences into two vectors, and then uses a classifier over the vector combination to label the relationship between these two sentences as that of entailment, contradiction, or neural. Our Shortcut-Stacked sentence encoders achieve strong improvements over existing encoders on matched and mismatched multi-domain natural language inference (top non-ensemble single-model result in the EMNLP RepEval 2017 Shared Task (Nangia et al., 2017)). Moreover, they achieve the new state-of-the-art encoding result on the original SNLI dataset (Bowman et al., 2015).

Figures (1)

• Figure 1: Our encoder's architecture: stacked biLSTM with shortcut connections and fine-tuning.