Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Contextual Information for Effective Entity Salience Detection

Rajarshi Bhowmik, Marco Ponza, Atharva Tendle, Anant Gupta, Rebecca Jiang, Xingyu Lu, Qian Zhao, Daniel Preotiuc-Pietro

TL;DR

This work investigates how to detect salient entities in text by fine-tuning medium-sized transformers with a cross-encoder architecture that explicitly encodes positional information around entity mentions. The method, which marks entity mentions and uses decile position embeddings, significantly outperforms feature-based baselines across four benchmarks, with substantial improvements in $F1$ scores. Analyses show the value of incorporating multiple inferred mentions and contextual cues, while zero-shot prompting of instruction-tuned LLMs underperforms, underscoring the task's uniqueness. The findings advance entity-centric understanding and have practical implications for search, ranking, and summarization by providing a robust, context-aware salience signal.

Abstract

In text documents such as news articles, the content and key events usually revolve around a subset of all the entities mentioned in a document. These entities, often deemed as salient entities, provide useful cues of the aboutness of a document to a reader. Identifying the salience of entities was found helpful in several downstream applications such as search, ranking, and entity-centric summarization, among others. Prior work on salient entity detection mainly focused on machine learning models that require heavy feature engineering. We show that fine-tuning medium-sized language models with a cross-encoder style architecture yields substantial performance gains over feature engineering approaches. To this end, we conduct a comprehensive benchmarking of four publicly available datasets using models representative of the medium-sized pre-trained language model family. Additionally, we show that zero-shot prompting of instruction-tuned language models yields inferior results, indicating the task's uniqueness and complexity.

Leveraging Contextual Information for Effective Entity Salience Detection

TL;DR

This work investigates how to detect salient entities in text by fine-tuning medium-sized transformers with a cross-encoder architecture that explicitly encodes positional information around entity mentions. The method, which marks entity mentions and uses decile position embeddings, significantly outperforms feature-based baselines across four benchmarks, with substantial improvements in scores. Analyses show the value of incorporating multiple inferred mentions and contextual cues, while zero-shot prompting of instruction-tuned LLMs underperforms, underscoring the task's uniqueness. The findings advance entity-centric understanding and have practical implications for search, ranking, and summarization by providing a robust, context-aware salience signal.

Abstract

In text documents such as news articles, the content and key events usually revolve around a subset of all the entities mentioned in a document. These entities, often deemed as salient entities, provide useful cues of the aboutness of a document to a reader. Identifying the salience of entities was found helpful in several downstream applications such as search, ranking, and entity-centric summarization, among others. Prior work on salient entity detection mainly focused on machine learning models that require heavy feature engineering. We show that fine-tuning medium-sized language models with a cross-encoder style architecture yields substantial performance gains over feature engineering approaches. To this end, we conduct a comprehensive benchmarking of four publicly available datasets using models representative of the medium-sized pre-trained language model family. Additionally, we show that zero-shot prompting of instruction-tuned language models yields inferior results, indicating the task's uniqueness and complexity.
Paper Structure (33 sections, 1 equation, 7 figures, 7 tables)

This paper contains 33 sections, 1 equation, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Definition
  4. Methods
  5. Cross-encoder
  6. Encoding
  7. Position Encoding
  8. Scoring
  9. Optimization
  10. Datasets
  11. NYT-Salience
  12. WN-Salience
  13. SEL
  14. EntSUM
  15. Data Enrichment with Inferred Mentions
  16. ...and 18 more sections

Figures (7)

  • Figure 1: An example of a document with salient and non-salient entities. Entity mentions are highlighted in text.
  • Figure 2: Graphical representation of the cross-encoder architecture with decile position encoding.
  • Figure 3: Stratified analysis across models and datasets.
  • Figure 4: Instruction for zero-shot prompting of LLaMa 2-Chat model.
  • Figure 5: Instruction for zero-shot prompting of Flan-UL2 model.
  • ...and 2 more figures