Reading Comprehension using Entity-based Memory Network

Abstract

This paper introduces a novel neural network model for question answering, the \emph{entity-based memory network}. It enhances neural networks' ability of representing and calculating information over a long period by keeping records of entities contained in text. The core component is a memory pool which comprises entities' states. These entities' states are continuously updated according to the input text. Questions with regard to the input text are used to search the memory pool for related entities and answers are further predicted based on the states of retrieved entities. Compared with previous memory network models, the proposed model is capable of handling fine-grained information and more sophisticated relations based on entities. We formulated several different tasks as question answering problems and tested the proposed model. Experiments reported satisfying results.

Figures (5)

• Figure 1: Comparison of the memory network and the proposed entity-based memory network model. Sentences are decomposed into entities and then stored in the memory for later retrieval.
• Figure 2: An Example from bAbI, a toy dataset for question answering weston2015towards.
• Figure 3: Architecture of the entity-based memory network. The model is divided into four modules which are shown in the figure using squares.
• Figure 4: the Generalization Module: Using $S_1$ as an example, the autoencoder is used to convert the sentence into a vector $\hbox{\boldmath$S_1$}$ and the entities contained in $S_1$ are used to reconstruct the sentence vector.
• Figure 5: the Output Feature Module: In each iteration, entities are assigned different scores which indicate their importance in constructing the output feature vector.