Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation

TL;DR

Environment-based Validation and Loss-based Sampling (EVaLS) effectively achieves group robustness, showing that group annotation is not necessary even for validation, marking a new chapter in the robustness of trained models against spurious correlation.

Abstract

Classifiers trained with Empirical Risk Minimization (ERM) tend to rely on attributes that have high spurious correlation with the target. This can degrade the performance on underrepresented (or 'minority') groups that lack these attributes, posing significant challenges for both out-of-distribution generalization and fairness objectives. Many studies aim to enhance robustness to spurious correlation, but they sometimes depend on group annotations for training. Additionally, a common limitation in previous research is the reliance on group-annotated validation datasets for model selection. This constrains their applicability in situations where the nature of the spurious correlation is not known, or when group labels for certain spurious attributes are not available. To enhance model robustness with minimal group annotation assumptions, we propose Environment-based Validation and Loss-based Sampling (EVaLS). It uses the losses from an ERM-trained model to construct a balanced dataset of high-loss and low-loss samples, mitigating group imbalance in data. This significantly enhances robustness to group shifts when equipped with a simple post-training last layer retraining. By using environment inference methods to create diverse environments with correlation shifts, EVaLS can potentially eliminate the need for group annotation in validation data. In this context, the worst environment accuracy acts as a reliable surrogate throughout the retraining process for tuning hyperparameters and finding a model that performs well across diverse group shifts. EVaLS effectively achieves group robustness, showing that group annotation is not necessary even for validation. It is a fast, straightforward, and effective approach that reaches near-optimal worst group accuracy without needing group annotations, marking a new chapter in the robustness of trained models against spurious correlation.

Figures (9)

• Figure 1: Overview of the proposed approach. (a) We randomly split the dataset $\mathcal{D}$ into $\mathcal{D}^\text{Tr}$, $\mathcal{D}^\text{MS}$, $\mathcal{D}^\text{LL}$ and $\mathcal{D}^\text{Te}$. We train the initial classifier on $\mathcal{D}^\text{Tr}$ with empirical risk minimization (ERM). Alternatively, we can assume that an ERM-trained model is given. (b) An environment inference method is utilized to infer diverse environments for each class of $\mathcal{D}^\text{MS}$. (c) We evaluate $\mathcal{D}^\text{LL}$ samples on the initial ERM classifier and sort high-loss and low-loss samples of each class for loss-based sampling. (d) Finally, we perform last-layer retraining on the loss-based selected samples $\mathcal{D}^\text{Bal}$. Each retraining setting (e.g. different $k$ for loss-based sampling) is validated based on the worst accuracy of the inferred environments. Note that majority and minority groups are shown with dark and light colors for better visualization, but are not known in our setting.
• Figure 2: The percentage of samples with the highest (lowest) losses across various thresholds that belong to the minority (majority) group within different classes in $\mathcal{D}^\text{LL}$ for (a) the Waterbirds and (b) CelebA datasets. Minority group samples are more prevalent among high-loss samples, while majority group samples dominate the low-loss areas. The error bars are calculated across three ERM models.
• Figure 3: (a) If all spurious attributes in a dataset are known, they can be utilized to fit a classifier that captures the essential attributes. (b) In the absence of knowledge about all spurious attributes, the model would depend on them for classification, leading to incorrect classification of minority samples. (c) If some spurious attribute is unknown (Spurious 2), the model becomes robust only to the known spurious correlations (Spurious 1), but it still underperforms on minority samples.
• Figure 4: (a) The Dominoes-CMF dataset, which contains two spurious attributes. (b) Performance on Dominoes-CMF is measured by worst-group accuracy across varying levels of correlation between the target label and the unknown spurious attribute (color). Lower reliance on available group annotations (based on known spurious attributes, i.e., style) results in higher robustness to both attributes. The performance gap between EVaLS and EVaLS-GL with lower group supervision compared to DFR DFR increases with higher correlations. The oracle uses DFR DFR with complete group information regarding both attributes.
• Figure 5: (a) Illustration of proportion density difference \ref{['def:proportional-density-diff-app']}, (b) equation of $tail_L(\alpha)=tail_R(\beta)$

Theorems & Definitions (13)

• Proposition 3.1
• Definition 3.1: Proportional Density Difference
• Lemma D.1
• proof
• Definition D.1: Proportional Density Difference
• Definition D.2: Tail Proportional Density Difference
• Corollary D.1
• Proposition D.1
• Lemma D.2
• proof