Characterizing the Multiclass Learnability of Forgiving 0-1 Loss Functions
Jacob Trauger, Tyson Trauger, Ambuj Tewari
TL;DR
The paper tackles PAC-learnability of forgiving $0$-$1$ loss in finite-label multiclass classification by introducing the Generalized Natarajan Dimension $GNdim(\mathcal{H},\ell)$, a dimension based on the Natarajan framework that captures the loss structure when indiscernibility does not strictly align with label equality. It proves a central theorem: a learning problem $(\mathcal{X},\mathcal{Y},\mathcal{H},\ell)$ is PAC-learnable if and only if $GNdim(\mathcal{H},\ell)<\infty$, via a reduction to an equivalent quotient problem on $\mathcal{Y}^C$ and a No-Free-Lunch-style argument for necessity, together with a uniform convergence-based sufficiency. Corollaries show that $GNdim(\mathcal{H},\ell)=Ndim(\mathcal{H}^C)$, linking forgiving loss learnability to the classical Natarajan dimension in the quotient space, and yielding a characterization for set learning through set-valued feedback. The work also demonstrates that $GNdim$ and the traditional Natarajan dimension can diverge, motivating future work on relaxing assumptions, extending to infinite label spaces, and quantifying learning rates under forgiving losses.
Abstract
In this paper we will give a characterization of the learnability of forgiving 0-1 loss functions in the finite label multiclass setting. To do this, we create a new combinatorial dimension that is based off of the Natarajan Dimension and we show that a hypothesis class is learnable in our setting if and only if this Generalized Natarajan Dimension is finite. We also show a connection to learning with set-valued feedback. Through our results we show that the learnability of a set learning problem is characterized by the Natarajan Dimension.