Multigroup Robustness

TL;DR

This work introduces multigroup robustness, a fine-grained notion that ensures a learning model’s degradation in predictions on any subpopulation is controlled by the amount of corruption within that subpopulation. It connects multigroup robustness to multiaccuracy and provides a practical post-processing approach that augments arbitrary learners with both multigroup robustness and multiaccuracy guarantees while preserving accuracy. The authors prove that multiaccuracy can yield multigroup robustness under distribution shift, and, with uniform convergence, extend robustness to stronger adversaries; they also show lower bounds establishing MA as a necessary property for non-trivial MR. Empirically, standard models exhibit vulnerability to simple subpopulation attacks, while the proposed post-processing method restores robustness without sacrificing performance, illustrating the practical impact for deployment in diverse subpopulations. Overall, the work bridges fairness and robustness by leveraging MA-based guarantees to achieve subgroup-protected robustness in realistic, non-i.i.d. data settings.

Abstract

To address the shortcomings of real-world datasets, robust learning algorithms have been designed to overcome arbitrary and indiscriminate data corruption. However, practical processes of gathering data may lead to patterns of data corruption that are localized to specific partitions of the training dataset. Motivated by critical applications where the learned model is deployed to make predictions about people from a rich collection of overlapping subpopulations, we initiate the study of multigroup robust algorithms whose robustness guarantees for each subpopulation only degrade with the amount of data corruption inside that subpopulation. When the data corruption is not distributed uniformly over subpopulations, our algorithms provide more meaningful robustness guarantees than standard guarantees that are oblivious to how the data corruption and the affected subpopulations are related. Our techniques establish a new connection between multigroup fairness and robustness.

Figures (13)

• Figure 1: Intuitive illustration of multigroup robustness: for every group $C$, if points within the group are not modified, a multigroup robust algorithm produces a predictor that achieves marginal mean consistency with the clean data predictor (See Definition \ref{['def:multigroup-robust']}).
• Figure 2: The effect of label change (0 to 1) in White male group on other subpopulations. For ${\operatorname{\mathsf{MA-err}}}$ (closer to 0 is better), the base models (Clf) are susceptible to noise other groups, Algorithm \ref{['alg:ma-empirical']} produces multigroup robust predictors (Clf-PP).
• Figure 3: The effect of targeting the White female subgroup when only data addition from the White male subgroup is allowed. Multigroup robust predictors (Clf-PP) maintain a consistently low ${\operatorname{\mathsf{MA-err}}}$ and high accuracy as more corrupted data is injected.
• Figure 4: Effect of label change in the white male group on other groups with a Logistic Regression Classifier. As the amount of noise increases, the ${\operatorname{\mathsf{MA-err}}}$ of the resulting predictor grows away from 0. The direction depends on whether the shift in label is from 0 to 1 or from 1 to 0.
• Figure 5: Effect of label change in the white female group on other groups with a Logistic Regression Classifier. In this group, flipping the labels has a large effect on the Black population.

Theorems & Definitions (36)

• Definition 4.1: Binary-label Multigroup Robustness
• Definition 4.2: Binary-label Multigroup Robustness to Distribution Shift
• Definition 5.1: Multiaccuracy hebert2018multicalibration
• Definition 5.2: Multiaccurate Learning Algorithm
• Lemma 5.3: Robustness from MA
• Lemma 5.4
• Definition 5.5: Empirically Multiaccurate Learning Algorithm
• Lemma 5.6: Pointwise Robustness from Empirical MA
• Definition 5.7
• Theorem 5.8