Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains

Yu Zhang, Yunyi Zhang, Yanzhen Shen, Yu Deng, Lucian Popa, Larisa Shwartz, ChengXiang Zhai, Jiawei Han

TL;DR

This work addresses seed-guided fine-grained entity typing in science and engineering by proposing SEType, a two-phase framework that first enriches a small set of seed entities per type using a large unlabeled corpus and then trains a cross-encoder entailment model on pseudo-labeled samples to classify both seen and unseen types. The entity enrichment phase computes contextualized embeddings from a PLM, enforces mutual exclusivity across types, and iteratively expands seed sets to improve coverage while controlling noise. The entailment phase treats context as a premise and each type, including unseen ones, as a hypothesis, training with a contrastive objective and performing inference by selecting the hypothesis with the highest entailment score. Experiments on StackOverflowNER and Cybersecurity datasets show SEType outperforms zero-shot and few-shot baselines, with ablations confirming the value of enrichment, context, and carefully chosen hyperparameters for reliable open-set typing in domain-specific text.

Abstract

Accurately typing entity mentions from text segments is a fundamental task for various natural language processing applications. Many previous approaches rely on massive human-annotated data to perform entity typing. Nevertheless, collecting such data in highly specialized science and engineering domains (e.g., software engineering and security) can be time-consuming and costly, without mentioning the domain gaps between training and inference data if the model needs to be applied to confidential datasets. In this paper, we study the task of seed-guided fine-grained entity typing in science and engineering domains, which takes the name and a few seed entities for each entity type as the only supervision and aims to classify new entity mentions into both seen and unseen types (i.e., those without seed entities). To solve this problem, we propose SEType which first enriches the weak supervision by finding more entities for each seen type from an unlabeled corpus using the contextualized representations of pre-trained language models. It then matches the enriched entities to unlabeled text to get pseudo-labeled samples and trains a textual entailment model that can make inferences for both seen and unseen types. Extensive experiments on two datasets covering four domains demonstrate the effectiveness of SEType in comparison with various baselines.

Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains

TL;DR

This work addresses seed-guided fine-grained entity typing in science and engineering by proposing SEType, a two-phase framework that first enriches a small set of seed entities per type using a large unlabeled corpus and then trains a cross-encoder entailment model on pseudo-labeled samples to classify both seen and unseen types. The entity enrichment phase computes contextualized embeddings from a PLM, enforces mutual exclusivity across types, and iteratively expands seed sets to improve coverage while controlling noise. The entailment phase treats context as a premise and each type, including unseen ones, as a hypothesis, training with a contrastive objective and performing inference by selecting the hypothesis with the highest entailment score. Experiments on StackOverflowNER and Cybersecurity datasets show SEType outperforms zero-shot and few-shot baselines, with ablations confirming the value of enrichment, context, and carefully chosen hyperparameters for reliable open-set typing in domain-specific text.

Abstract

Accurately typing entity mentions from text segments is a fundamental task for various natural language processing applications. Many previous approaches rely on massive human-annotated data to perform entity typing. Nevertheless, collecting such data in highly specialized science and engineering domains (e.g., software engineering and security) can be time-consuming and costly, without mentioning the domain gaps between training and inference data if the model needs to be applied to confidential datasets. In this paper, we study the task of seed-guided fine-grained entity typing in science and engineering domains, which takes the name and a few seed entities for each entity type as the only supervision and aims to classify new entity mentions into both seen and unseen types (i.e., those without seed entities). To solve this problem, we propose SEType which first enriches the weak supervision by finding more entities for each seen type from an unlabeled corpus using the contextualized representations of pre-trained language models. It then matches the enriched entities to unlabeled text to get pseudo-labeled samples and trains a textual entailment model that can make inferences for both seen and unseen types. Extensive experiments on two datasets covering four domains demonstrate the effectiveness of SEType in comparison with various baselines.
Paper Structure (17 sections, 8 equations, 3 figures, 3 tables)

This paper contains 17 sections, 8 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. The SEType Framework
  4. Entity Enrichment
  5. Entailment Model Training
  6. Inference
  7. Experiments
  8. Datasets
  9. Compared Methods
  10. Implementation and Hyperparameters
  11. Evaluation Metrics
  12. Performance Comparison
  13. Hyperparameter Analyses
  14. Related Work
  15. Zero-shot and Few-shot Entity Typing
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Overview of the SEType framework.
  • Figure 2: Performance of SEType with different window sizes of context sentences (the sentence containing the entity mention and its $\pm c$ sentences are fed into the PLM) on StackOverflow and GitHub. Considering $\pm 1$ sentences is always better than focusing on the mention sentence alone.
  • Figure 3: Performance of SEType with different numbers of enriched entities per type (i.e., $|\mathcal{E}_i^+|$) on StackOverflow and NVD.

Theorems & Definitions (1)

  • Definition 1: Problem Definition