Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Global Calibration Strengthens Multiaccuracy

Sílvia Casacuberta, Parikshit Gopalan, Varun Kanade, Omer Reingold

TL;DR

This work analyzes the power of multiaccuracy as a learning primitive and shows that, while multiaccuracy alone can be weak, pairing it with global calibration yields strong learning guarantees and optimal hardcore distributions. The authors establish precise results: calibrated multiaccuracy enables strong agnostic learning; multiaccuracy alone yields restricted weak learning only when the target correlation exceeds $1/2$; and combining calibration with weighted multiaccuracy achieves hardcore densities of $2\delta$ with favorable oracle complexity $q=O(1/(\varepsilon^2\delta^2))$, matching the best-known density bounds while staying computationally efficient. The results illuminate the complementary roles of multiaccuracy and calibration and explain why their combination yields substantially stronger guarantees than either notion alone. Practically, this suggests that implementing calibration alongside standard multiaccuracy can unlock the strengths of multicalibration at lower cost, with implications for fair prediction, robustness, and hardness amplification in learning systems.

Abstract

Multiaccuracy and multicalibration are multigroup fairness notions for prediction that have found numerous applications in learning and computational complexity. They can be achieved from a single learning primitive: weak agnostic learning. Here we investigate the power of multiaccuracy as a learning primitive, both with and without the additional assumption of calibration. We find that multiaccuracy in itself is rather weak, but that the addition of global calibration (this notion is called calibrated multiaccuracy) boosts its power substantially, enough to recover implications that were previously known only assuming the stronger notion of multicalibration. We give evidence that multiaccuracy might not be as powerful as standard weak agnostic learning, by showing that there is no way to post-process a multiaccurate predictor to get a weak learner, even assuming the best hypothesis has correlation $1/2$. Rather, we show that it yields a restricted form of weak agnostic learning, which requires some concept in the class to have correlation greater than $1/2$ with the labels. However, by also requiring the predictor to be calibrated, we recover not just weak, but strong agnostic learning. A similar picture emerges when we consider the derivation of hardcore measures from predictors satisfying multigroup fairness notions. On the one hand, while multiaccuracy only yields hardcore measures of density half the optimal, we show that (a weighted version of) calibrated multiaccuracy achieves optimal density. Our results yield new insights into the complementary roles played by multiaccuracy and calibration in each setting. They shed light on why multiaccuracy and global calibration, although not particularly powerful by themselves, together yield considerably stronger notions.

How Global Calibration Strengthens Multiaccuracy

TL;DR

This work analyzes the power of multiaccuracy as a learning primitive and shows that, while multiaccuracy alone can be weak, pairing it with global calibration yields strong learning guarantees and optimal hardcore distributions. The authors establish precise results: calibrated multiaccuracy enables strong agnostic learning; multiaccuracy alone yields restricted weak learning only when the target correlation exceeds ; and combining calibration with weighted multiaccuracy achieves hardcore densities of with favorable oracle complexity , matching the best-known density bounds while staying computationally efficient. The results illuminate the complementary roles of multiaccuracy and calibration and explain why their combination yields substantially stronger guarantees than either notion alone. Practically, this suggests that implementing calibration alongside standard multiaccuracy can unlock the strengths of multicalibration at lower cost, with implications for fair prediction, robustness, and hardness amplification in learning systems.

Abstract

Multiaccuracy and multicalibration are multigroup fairness notions for prediction that have found numerous applications in learning and computational complexity. They can be achieved from a single learning primitive: weak agnostic learning. Here we investigate the power of multiaccuracy as a learning primitive, both with and without the additional assumption of calibration. We find that multiaccuracy in itself is rather weak, but that the addition of global calibration (this notion is called calibrated multiaccuracy) boosts its power substantially, enough to recover implications that were previously known only assuming the stronger notion of multicalibration. We give evidence that multiaccuracy might not be as powerful as standard weak agnostic learning, by showing that there is no way to post-process a multiaccurate predictor to get a weak learner, even assuming the best hypothesis has correlation . Rather, we show that it yields a restricted form of weak agnostic learning, which requires some concept in the class to have correlation greater than with the labels. However, by also requiring the predictor to be calibrated, we recover not just weak, but strong agnostic learning. A similar picture emerges when we consider the derivation of hardcore measures from predictors satisfying multigroup fairness notions. On the one hand, while multiaccuracy only yields hardcore measures of density half the optimal, we show that (a weighted version of) calibrated multiaccuracy achieves optimal density. Our results yield new insights into the complementary roles played by multiaccuracy and calibration in each setting. They shed light on why multiaccuracy and global calibration, although not particularly powerful by themselves, together yield considerably stronger notions.

Paper Structure

This paper contains 42 sections, 28 theorems, 113 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Calibrated Multiaccuracy.
  3. Computational learning.
  4. Hardcore set constructions.
  5. The three tiers of multigroup fairness.
  6. The case for calibrated multiaccuracy.
  7. Overview of our Results
  8. Multigroup fairness notions.
  9. Agnostic learning.
  10. Multigroup fairness and agnostic learning.
  11. Agnostic Learning and Multiaccuracy: With and Without Calibration
  12. When multiaccuracy does not imply agnostic learning.
  13. When multiaccuracy implies (restricted) weak agnostic learning.
  14. Calibrated multiaccuracy gives strong agnostic learning.
  15. Hardcore Sets of Optimal Density
  16. ...and 27 more sections

Key Result

Theorem 2.1

There exists $q = \mathop{\mathrm{poly}}\nolimits(1/\varepsilon, 1/\delta)$ such that if $g: \mathcal{X} \rightarrow \{0,1\}$ is $(\mathcal{C}_{q}, \delta)$-hard on $\mathcal{D}_{\mathcal{X}}$, then there is a measure $\mu$ satisfying:

Figures (3)

  • Figure 1: Three tiers of multigroup fairness notions.
  • Figure 2: The spectrum of weak agnostic learning.
  • Figure 3: Weights assigned by $\mu_{\mathsf{TTV}}$ and $\mu_{\mathsf{Max}}$ on each region of $\mathcal{X}$. The two measures nearly agree on the easy blue region, but $\mu_{\mathsf{Max}}$ is twice $\mu_{\mathsf{TTV}}$ on the hard red region.

Theorems & Definitions (55)

  • Theorem 2.1: IHCL informal statement, imp95hol05
  • Theorem 2.2: IHCL from calibration and weighted multiaccuracy
  • Theorem 2.3
  • Definition 3.1
  • Definition 3.2: Weak agnostic learner
  • Definition 3.3: Strong agnostic learner
  • Theorem 3.4: Agnostic Boosting, kk09
  • Theorem 3.5
  • proof
  • Definition 3.6
  • ...and 45 more