Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neyman-Pearson multiclass classification under label noise via empirical likelihood

Qiong Zhang, Qinglong Tian, Pengfei Li

Abstract

In many classification problems, the costs of misclassifying observations from different classes can be highly unequal. The Neyman-Pearson multiclass classification (NPMC) framework addresses this issue by minimizing a weighted misclassification risk while imposing upper bounds on class-specific error probabilities. Existing NPMC methods typically assume that training labels are correctly observed. In practice, however, labels are often corrupted due to measurement error or annotation, and the effect of such label noise on NPMC procedures remains largely unexplored. We study the NPMC problem when only noisy labels are available in the training data. We propose an empirical likelihood (EL)-based method that relates the distributions of noisy and true labels through an exponential tilting density ratio model. The resulting maximum EL estimators recover the class proportions and posterior probabilities of the clean labels required for error control. We establish consistency, asymptotic normality, and optimal convergence rates for these estimators. Under mild conditions, the resulting classifier satisfies NP oracle inequalities with respect to the true labels asymptotically. An expectation-maximization algorithm computes the maximum EL estimators. Simulations show that the proposed method performs comparably to the oracle classifier under clean labels and substantially improves over procedures that ignore label noise.

Neyman-Pearson multiclass classification under label noise via empirical likelihood

Abstract

In many classification problems, the costs of misclassifying observations from different classes can be highly unequal. The Neyman-Pearson multiclass classification (NPMC) framework addresses this issue by minimizing a weighted misclassification risk while imposing upper bounds on class-specific error probabilities. Existing NPMC methods typically assume that training labels are correctly observed. In practice, however, labels are often corrupted due to measurement error or annotation, and the effect of such label noise on NPMC procedures remains largely unexplored. We study the NPMC problem when only noisy labels are available in the training data. We propose an empirical likelihood (EL)-based method that relates the distributions of noisy and true labels through an exponential tilting density ratio model. The resulting maximum EL estimators recover the class proportions and posterior probabilities of the clean labels required for error control. We establish consistency, asymptotic normality, and optimal convergence rates for these estimators. Under mild conditions, the resulting classifier satisfies NP oracle inequalities with respect to the true labels asymptotically. An expectation-maximization algorithm computes the maximum EL estimators. Simulations show that the proposed method performs comparably to the oracle classifier under clean labels and substantially improves over procedures that ignore label noise.
Paper Structure (51 sections, 19 theorems, 216 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 51 sections, 19 theorems, 216 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem setting
  3. Empirical likelihood based estimation under label noise
  4. Exponential tilting and density ratio model
  5. Identifiability
  6. Empirical likelihood based inference
  7. Statistical guarantees
  8. EM algorithm for profile EL maximization
  9. Neyman-Pearson binary classification
  10. Revisiting NP binary classification with clean labels
  11. NP binary classification with noisy labels with statistical guarantees
  12. Simulation study
  13. Neyman-Pearson multiclass classification
  14. Revisiting NPMC with clean labels
  15. NPMC with noisy labels with statistical guarantees
  16. ...and 36 more sections

Key Result

Proposition 2.1

Let $\widetilde{w}_l^* = {\mathbb{P}}(\widetilde{Y}=l)$ and $w_k^* = {\mathbb{P}}(Y=k)$ denote the noisy and true class proportions, respectively. Let $\pi_k^*(x) = {\mathbb{P}}(Y=k| X=x)$ and $\widetilde{\pi}_l^*(x) ={\mathbb{P}}(\widetilde{Y}=l| X=x)$. Define: Then, under Assumption assumption:instance_independent_noise, the following relationships hold:

Figures (4)

  • Figure 1: Violin plots (top to bottom) show the misclassification errors for Class 0, Class 1, and the objective function value under Case (a), computed over $R = 500$ repetitions. Results compare different methods across varying sample sizes ($n$) and noise levels ($\eta$). The dashed black lines in the first two rows mark the target misclassification errors.
  • Figure 2: Mean squared error (MSE) of regression coefficients for different estimators: Oracle (dash-dotted line), our method (solid line), and Vanilla (dashed line). Results are shown under varying noise levels (colors) and sample sizes $n$, with cases (a)–(c) displayed from left to right.
  • Figure 3: Violin plots (top to bottom) show the misclassification errors for Class 0, Class 3, and the objective function value under Case (b), computed over $R = 500$ repetitions. Results compare different methods across varying sample sizes ($n$) and noise levels ($\eta$). The dashed black lines in the first two rows mark the target misclassification errors.
  • Figure 4: Violin plots (top to bottom) show the misclassification errors for Class 0, Class 1, and the objective function value under Case (c), computed over $R = 500$ repetitions. Results compare different methods across varying sample sizes ($n$) and noise levels ($\eta$). The dashed black lines in the first two rows mark the target misclassification errors.

Theorems & Definitions (50)

  • Proposition 2.1: Distributional relationships under label noise
  • Remark 2.1
  • Remark 3.1: Basis function selection
  • Example 3.1: Algebraic unidentifiability of true distributions
  • Example 3.2: Unidentifiability due to linearly dependent features
  • Theorem 3.1: Identifiability
  • Theorem 3.2: Estimator rate of convergence
  • Proposition 3.1: Convergence of EM algorithm
  • Remark 3.2: Incorporating prior knowledge
  • Theorem 4.1: Error control guarantee
  • ...and 40 more