Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modelling Interaction of Sentence Pair with coupled-LSTMs

Pengfei Liu, Xipeng Qiu, Xuanjing Huang

TL;DR

The paper tackles sentence-pair semantic matching by shifting from independent sentence encoders to coupled-LSTMs that model strong, bidirectional interactions. It introduces two variants, LC-LSTMs and TC-LSTMs, with four-directional processing and an aggregation/dynamic-pooling scheme to capture multi-granularity interactions end-to-end. Empirical results on large-scale SNLI entailment and Yahoo! Answers question-answer tasks show the proposed models achieve state-of-the-art or competitive performance with fewer parameters, and neuron-level analyses reveal interpretable interaction patterns. The work suggests future enhancements via gating mechanisms to further deepen depth- interaction and performance.

Abstract

Recently, there is rising interest in modelling the interactions of two sentences with deep neural networks. However, most of the existing methods encode two sequences with separate encoders, in which a sentence is encoded with little or no information from the other sentence. In this paper, we propose a deep architecture to model the strong interaction of sentence pair with two coupled-LSTMs. Specifically, we introduce two coupled ways to model the interdependences of two LSTMs, coupling the local contextualized interactions of two sentences. We then aggregate these interactions and use a dynamic pooling to select the most informative features. Experiments on two very large datasets demonstrate the efficacy of our proposed architecture and its superiority to state-of-the-art methods.

Modelling Interaction of Sentence Pair with coupled-LSTMs

TL;DR

The paper tackles sentence-pair semantic matching by shifting from independent sentence encoders to coupled-LSTMs that model strong, bidirectional interactions. It introduces two variants, LC-LSTMs and TC-LSTMs, with four-directional processing and an aggregation/dynamic-pooling scheme to capture multi-granularity interactions end-to-end. Empirical results on large-scale SNLI entailment and Yahoo! Answers question-answer tasks show the proposed models achieve state-of-the-art or competitive performance with fewer parameters, and neuron-level analyses reveal interpretable interaction patterns. The work suggests future enhancements via gating mechanisms to further deepen depth- interaction and performance.

Abstract

Recently, there is rising interest in modelling the interactions of two sentences with deep neural networks. However, most of the existing methods encode two sequences with separate encoders, in which a sentence is encoded with little or no information from the other sentence. In this paper, we propose a deep architecture to model the strong interaction of sentence pair with two coupled-LSTMs. Specifically, we introduce two coupled ways to model the interdependences of two LSTMs, coupling the local contextualized interactions of two sentences. We then aggregate these interactions and use a dynamic pooling to select the most informative features. Experiments on two very large datasets demonstrate the efficacy of our proposed architecture and its superiority to state-of-the-art methods.

Paper Structure

This paper contains 35 sections, 10 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Weak interaction Models
  3. Semi-interaction Models
  4. Strong Interaction Models
  5. Sentence Modelling with LSTM
  6. Coupled-LSTMs for Strong Sentence Interaction
  7. Loosely Coupled-LSTMs (LC-LSTMs)
  8. Tightly Coupled-LSTMs (TC-LSTMs)
  9. Analysis of Two Proposed Models
  10. Interaction Between Temporal Dimensions
  11. Interaction Between Depth Dimension
  12. Comparison of LC-LSTMs and word-by-word Attention LSTMs
  13. End-to-End Architecture for Sentence Matching
  14. Embedding Layer
  15. Stacked Coupled-LSTMs Layers
  16. ...and 20 more sections

Figures (3)

  • Figure 1: Four different coupled-LSTMs.
  • Figure 2: Architecture of coupled-LSTMs for sentence-pair encoding. Inputs are fed to four C-LSTMs followed by an aggregation layer. Blue cuboids represent different contextual information from four directions.
  • Figure 3: Illustration of two interpretable neurons and some word-pairs capture by these neurons. The darker patches denote the corresponding activations are higher.