Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constraint Multi-class Positive and Unlabeled Learning for Distantly Supervised Named Entity Recognition

Yuzhe Zhang, Min Cen, Hong Zhang

TL;DR

The paper tackles distant supervision for NER by addressing incomplete labeling from dictionaries and KBs. It introduces CMPU, a constrained multi-class Positive-Unlabeled learning method that adds a constraint factor $\lambda$ to balance positive and unlabeled risks and prevent rapid deterioration of unlabeled risk. The authors provide theoretical guarantees, including a consistency bound and an estimation-error bound, and demonstrate substantial empirical gains over existing DS-NER methods on BC5CDR and CoNLL2003 while showing robustness to dictionary coverage. This work offers a principled, provably robust alternative for DS-NER with clear guidance on parameter tuning and practical applicability to real-world labeling noise.

Abstract

Distantly supervised named entity recognition (DS-NER) has been proposed to exploit the automatically labeled training data by external knowledge bases instead of human annotations. However, it tends to suffer from a high false negative rate due to the inherent incompleteness. To address this issue, we present a novel approach called \textbf{C}onstraint \textbf{M}ulti-class \textbf{P}ositive and \textbf{U}nlabeled Learning (CMPU), which introduces a constraint factor on the risk estimator of multiple positive classes. It suggests that the constraint non-negative risk estimator is more robust against overfitting than previous PU learning methods with limited positive data. Solid theoretical analysis on CMPU is provided to prove the validity of our approach. Extensive experiments on two benchmark datasets that were labeled using diverse external knowledge sources serve to demonstrate the superior performance of CMPU in comparison to existing DS-NER methods.

Constraint Multi-class Positive and Unlabeled Learning for Distantly Supervised Named Entity Recognition

TL;DR

The paper tackles distant supervision for NER by addressing incomplete labeling from dictionaries and KBs. It introduces CMPU, a constrained multi-class Positive-Unlabeled learning method that adds a constraint factor to balance positive and unlabeled risks and prevent rapid deterioration of unlabeled risk. The authors provide theoretical guarantees, including a consistency bound and an estimation-error bound, and demonstrate substantial empirical gains over existing DS-NER methods on BC5CDR and CoNLL2003 while showing robustness to dictionary coverage. This work offers a principled, provably robust alternative for DS-NER with clear guidance on parameter tuning and practical applicability to real-world labeling noise.

Abstract

Distantly supervised named entity recognition (DS-NER) has been proposed to exploit the automatically labeled training data by external knowledge bases instead of human annotations. However, it tends to suffer from a high false negative rate due to the inherent incompleteness. To address this issue, we present a novel approach called \textbf{C}onstraint \textbf{M}ulti-class \textbf{P}ositive and \textbf{U}nlabeled Learning (CMPU), which introduces a constraint factor on the risk estimator of multiple positive classes. It suggests that the constraint non-negative risk estimator is more robust against overfitting than previous PU learning methods with limited positive data. Solid theoretical analysis on CMPU is provided to prove the validity of our approach. Extensive experiments on two benchmark datasets that were labeled using diverse external knowledge sources serve to demonstrate the superior performance of CMPU in comparison to existing DS-NER methods.

Paper Structure

This paper contains 16 sections, 5 theorems, 56 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Distantly Supervised NER
  4. PU Learning
  5. Methodology
  6. Learning Setup
  7. CMPU Learning
  8. Theoretical Results
  9. Experiments
  10. Experimental Setup
  11. Baselines
  12. Experimental Results
  13. Main Results
  14. Impact of Constraint Factor $\lambda$
  15. Impact of Dictionary Size
  16. ...and 1 more sections

Key Result

Theorem 3.1

Assume that there exists a constant $\alpha$ such that $R_{U}^{-}(f) - \sum_{i=1}^{C} \pi_i R_{P_i}^{-}(f) - \lambda \sum_{i=1}^{C} \pi_i R^{+}_{P_i}(f) \geq \alpha$, and $\sup_{f \in \mathcal{H}} ||f||_{\infty} \leq C_f$ and $\sup_{|t| \leq C_f} \max_{0 \leq y \leq C} \ell(t, y) \leq C_{\ell}$, whe where

Figures (3)

  • Figure 1: An annotated example with distant supervision. The entity "Obama" of type PER is not recognized.
  • Figure 2: $F_1$ score curves of different constraint factor $\lambda$ on (a) BC5CDR(Dict) and (b) CoNLL2003(Dict) datasets.
  • Figure 3: Precision, recall and $F_1$ scores on BC5CDR ((a)-(c)) and CoNLL2003 ((d)-(f)) datasets under different dictionary sizes.

Theorems & Definitions (10)

  • Theorem 3.1: Consistency
  • Theorem 3.2: Estimation error bound
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • proof
  • proof