Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Seq2seq Coreference Resolution Using Entity Representations

Matt Grenander, Shay B. Cohen, Mark Steedman

TL;DR

This work tackles incremental coreference resolution with seq2seq methods by introducing an Entity-Centric compressed representation that retains only entity-related tokens and a short contextual window. By reorganizing and memory-mapping entity mentions, the approach dramatically reduces input size and memory usage while delivering near state-of-the-art performance on OntoNotes and achieving SOTA results on LitBank. Key findings show the method is feasible for incremental processing with a compression ratio around 1.8x and substantial memory savings, albeit with some recall trade-offs in challenging categories like named entities and definite noun phrases. The results demonstrate that token-level compression focused on entities can preserve coreference signals and offer practical benefits for real-time or dialogue-style applications. The work also analyzes error sources, augmentation strategies, and the role of dataset annotations (e.g., singleton handling) in incremental settings, outlining clear avenues for further improvement.

Abstract

Seq2seq coreference models have introduced a new paradigm for coreference resolution by learning to generate text corresponding to coreference labels, without requiring task-specific parameters. While these models achieve new state-of-the-art performance, they do so at the cost of flexibility and efficiency. In particular, they do not efficiently handle incremental settings such as dialogue, where text must processed sequentially. We propose a compressed representation in order to improve the efficiency of these methods in incremental settings. Our method works by extracting and re-organizing entity-level tokens, and discarding the majority of other input tokens. On OntoNotes, our best model achieves just 0.6 CoNLL F1 points below a full-prefix, incremental baseline while achieving a compression ratio of 1.8. On LitBank, where singleton mentions are annotated, it passes state-of-the-art performance. Our results indicate that discarding a wide portion of tokens in seq2seq resolvers is a feasible strategy for incremental coreference resolution.

Efficient Seq2seq Coreference Resolution Using Entity Representations

TL;DR

This work tackles incremental coreference resolution with seq2seq methods by introducing an Entity-Centric compressed representation that retains only entity-related tokens and a short contextual window. By reorganizing and memory-mapping entity mentions, the approach dramatically reduces input size and memory usage while delivering near state-of-the-art performance on OntoNotes and achieving SOTA results on LitBank. Key findings show the method is feasible for incremental processing with a compression ratio around 1.8x and substantial memory savings, albeit with some recall trade-offs in challenging categories like named entities and definite noun phrases. The results demonstrate that token-level compression focused on entities can preserve coreference signals and offer practical benefits for real-time or dialogue-style applications. The work also analyzes error sources, augmentation strategies, and the role of dataset annotations (e.g., singleton handling) in incremental settings, outlining clear avenues for further improvement.

Abstract

Seq2seq coreference models have introduced a new paradigm for coreference resolution by learning to generate text corresponding to coreference labels, without requiring task-specific parameters. While these models achieve new state-of-the-art performance, they do so at the cost of flexibility and efficiency. In particular, they do not efficiently handle incremental settings such as dialogue, where text must processed sequentially. We propose a compressed representation in order to improve the efficiency of these methods in incremental settings. Our method works by extracting and re-organizing entity-level tokens, and discarding the majority of other input tokens. On OntoNotes, our best model achieves just 0.6 CoNLL F1 points below a full-prefix, incremental baseline while achieving a compression ratio of 1.8. On LitBank, where singleton mentions are annotated, it passes state-of-the-art performance. Our results indicate that discarding a wide portion of tokens in seq2seq resolvers is a feasible strategy for incremental coreference resolution.
Paper Structure (27 sections, 4 equations, 5 figures, 7 tables)

This paper contains 27 sections, 4 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Seq2seq Coreference Resolution
  5. Full-Prefix Incremental Baseline
  6. Entity-Centric Representation
  7. Experiments
  8. Datasets
  9. Metrics
  10. Comparisons
  11. Non-Incremental
  12. Full-Prefix Incremental Baseline
  13. Entity-Centric
  14. Other Baselines
  15. Training
  16. ...and 12 more sections

Figures (5)

  • Figure 1: Top: An example of the Full-Prefix Incremental setting. The text in blue represents previously labelled chunks. The model will predict coreference clusters in the next chunk, enclosed by <target> and </target> tokens. Middle: An example from the Entity-Centric representation. Mentions from the same cluster are grouped together and labelled with their cluster identity. The fixed-length context, shown in blue, consists of previously labelled sentences as in the incremental setting. Bottom: The expected output for this sample.
  • Figure 2: Compression Ratio experiments on the OntoNotes validation set.
  • Figure 3: Mentions correctly predicted by the non-incremental model but missed by the Full-Prefix Incremental model, divided by syntactic category and string overlap with antecedent.
  • Figure 4: Results on the OntoNotes validation set after adding gold coreference links corresponding to specified syntactic categories. Non-Inc.=Non-Incremental, EM=Exact Match, PM=Partial Match, NE=Named Entities, Def.=Definite noun phrases.
  • Figure 5: An example error from the Entity-Centric model not made by the Full-Prefix Incremental baseline. The Entity-Centric model only stores the mention "us", without additional context, meaning the model cannot infer that "us" refers to "journalists from CNN". It subsequently marks "CNN" as a separate cluster.