Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Coreference Resolution with Pretrained Language Models: Bridging the Gap Between Syntax and Semantics

Xingzu Liu, Songhang deng, Mingbang Wang, Zhang Dong, Le Dai, Jiyuan Li, Ruilin Nong

TL;DR

This work tackles coreference resolution by bridging syntax and semantics through a framework that combines syntactic parsing with semantic role labeling and leverages pretrained language models to produce contextual embeddings. An attention-based decoder tunes coreferential links using integrated syntactic-semantic representations, yielding improved accuracy across diverse datasets. Comprehensive ablations demonstrate the additive benefits of each component, with full integration achieving the highest gains over traditional systems and strong baselines. The results underscore the practical value of syntactic-semantic fusion for refined referential understanding and its potential to enhance downstream NLP tasks.

Abstract

Large language models have made significant advancements in various natural language processing tasks, including coreference resolution. However, traditional methods often fall short in effectively distinguishing referential relationships due to a lack of integration between syntactic and semantic information. This study introduces an innovative framework aimed at enhancing coreference resolution by utilizing pretrained language models. Our approach combines syntax parsing with semantic role labeling to accurately capture finer distinctions in referential relationships. By employing state-of-the-art pretrained models to gather contextual embeddings and applying an attention mechanism for fine-tuning, we improve the performance of coreference tasks. Experimental results across diverse datasets show that our method surpasses conventional coreference resolution systems, achieving notable accuracy in disambiguating references. This development not only improves coreference resolution outcomes but also positively impacts other natural language processing tasks that depend on precise referential understanding.

Enhancing Coreference Resolution with Pretrained Language Models: Bridging the Gap Between Syntax and Semantics

TL;DR

This work tackles coreference resolution by bridging syntax and semantics through a framework that combines syntactic parsing with semantic role labeling and leverages pretrained language models to produce contextual embeddings. An attention-based decoder tunes coreferential links using integrated syntactic-semantic representations, yielding improved accuracy across diverse datasets. Comprehensive ablations demonstrate the additive benefits of each component, with full integration achieving the highest gains over traditional systems and strong baselines. The results underscore the practical value of syntactic-semantic fusion for refined referential understanding and its potential to enhance downstream NLP tasks.

Abstract

Large language models have made significant advancements in various natural language processing tasks, including coreference resolution. However, traditional methods often fall short in effectively distinguishing referential relationships due to a lack of integration between syntactic and semantic information. This study introduces an innovative framework aimed at enhancing coreference resolution by utilizing pretrained language models. Our approach combines syntax parsing with semantic role labeling to accurately capture finer distinctions in referential relationships. By employing state-of-the-art pretrained models to gather contextual embeddings and applying an attention mechanism for fine-tuning, we improve the performance of coreference tasks. Experimental results across diverse datasets show that our method surpasses conventional coreference resolution systems, achieving notable accuracy in disambiguating references. This development not only improves coreference resolution outcomes but also positively impacts other natural language processing tasks that depend on precise referential understanding.

Paper Structure

This paper contains 24 sections, 6 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Coreference Resolution Techniques
  4. Syntax-Semantics Integration
  5. Pretrained Language Models in NLP
  6. Methodology
  7. Coreference Resolution
  8. Syntactic and Semantic Integration
  9. Contextual Embedding Enhancement
  10. Experimental Setup
  11. Datasets
  12. Baselines
  13. Models
  14. Implements
  15. Experiments
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Syntactic structures with semantic on BERT structure.
  • Figure 2: Performance of various embedding extraction methods on coreference resolution tasks.
  • Figure 3: Comparison of various attention mechanisms implemented in the coreference resolution framework. The F1 scores reflect the efficacy of each mechanism, while the performance gain indicates the improvements over the baseline.
  • Figure 4: Comparison of referential relationship distinction performance between the baseline and the proposed model measured in F1 score (%).