Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Majority or Minority: Data Imbalance Learning Method for Named Entity Recognition

Sota Nemoto, Shunsuke Kitada, Hitoshi Iyatomi

TL;DR

This work tackles the data imbalance problem in named entity recognition (NER), where a long-tail distribution yields a dominant non-entity (O) class alongside many minority entity classes. It introduces Majority or Minority (MoM) learning, a simple loss augmentation that adds the loss computed only for O-class tokens to the conventional model loss, controlled by a single trade-off parameter $\lambda$. Empirical results across four English/Japanese NER datasets and two frameworks (sequential labeling and machine reading comprehension) show that MoM consistently improves minority entity performance without sacrificing O-class accuracy, outperforming weighted cross-entropy, focal loss, and dice loss baselines. The approach is language-, model-, framework-, and data-size-agnostic, offering a practical and robust solution to long-tail imbalances with broad applicability in NLP systems and downstream information extraction tasks.

Abstract

Data imbalance presents a significant challenge in various machine learning (ML) tasks, particularly named entity recognition (NER) within natural language processing (NLP). NER exhibits a data imbalance with a long-tail distribution, featuring numerous minority classes (i.e., entity classes) and a single majority class (i.e., O-class). This imbalance leads to misclassifications of the entity classes as the O-class. To tackle this issue, we propose a simple and effective learning method named majority or minority (MoM) learning. MoM learning incorporates the loss computed only for samples whose ground truth is the majority class into the loss of the conventional ML model. Evaluation experiments on four NER datasets (Japanese and English) showed that MoM learning improves prediction performance of the minority classes without sacrificing the performance of the majority class and is more effective than widely known and state-of-the-art methods. We also evaluated MoM learning using frameworks as sequential labeling and machine reading comprehension, which are commonly used in NER. Furthermore, MoM learning has achieved consistent performance improvements regardless of language or framework.

Majority or Minority: Data Imbalance Learning Method for Named Entity Recognition

TL;DR

This work tackles the data imbalance problem in named entity recognition (NER), where a long-tail distribution yields a dominant non-entity (O) class alongside many minority entity classes. It introduces Majority or Minority (MoM) learning, a simple loss augmentation that adds the loss computed only for O-class tokens to the conventional model loss, controlled by a single trade-off parameter . Empirical results across four English/Japanese NER datasets and two frameworks (sequential labeling and machine reading comprehension) show that MoM consistently improves minority entity performance without sacrificing O-class accuracy, outperforming weighted cross-entropy, focal loss, and dice loss baselines. The approach is language-, model-, framework-, and data-size-agnostic, offering a practical and robust solution to long-tail imbalances with broad applicability in NLP systems and downstream information extraction tasks.

Abstract

Data imbalance presents a significant challenge in various machine learning (ML) tasks, particularly named entity recognition (NER) within natural language processing (NLP). NER exhibits a data imbalance with a long-tail distribution, featuring numerous minority classes (i.e., entity classes) and a single majority class (i.e., O-class). This imbalance leads to misclassifications of the entity classes as the O-class. To tackle this issue, we propose a simple and effective learning method named majority or minority (MoM) learning. MoM learning incorporates the loss computed only for samples whose ground truth is the majority class into the loss of the conventional ML model. Evaluation experiments on four NER datasets (Japanese and English) showed that MoM learning improves prediction performance of the minority classes without sacrificing the performance of the majority class and is more effective than widely known and state-of-the-art methods. We also evaluated MoM learning using frameworks as sequential labeling and machine reading comprehension, which are commonly used in NER. Furthermore, MoM learning has achieved consistent performance improvements regardless of language or framework.
Paper Structure (22 sections, 6 equations, 4 figures, 9 tables)

This paper contains 22 sections, 6 equations, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sampling-based Method
  4. Cost-Sensitive Learning
  5. Proposed Method
  6. Notation
  7. Majority or Minority (MoM) Learning
  8. Experiments
  9. NER Frameworks
  10. Sequential labeling framework
  11. MRC framework
  12. Datasets
  13. Loss Functions
  14. WCE
  15. FL
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Concept of the MoM learning. The conventional loss function, $\mathcal{L}(\bm{y}^{(n)}, \bm{p}^{(n)})$ (e.g., cross-entropy loss), calculates the loss for all samples. In the MoM learning, $\mathcal{L}_{\mathrm{MoM}}(\bm{y}^{(n)}, \bm{p}^{(n)})$, the loss associated with the "major" $\mathcal{O}$-class is added to $\mathcal{L}(\bm{y}^{(n)}, \bm{p}^{(n)})$.
  • Figure 2: macro-F1
  • Figure 3: sentence-level
  • Figure 4: word-level