Rethinking Robustness in Machine Learning: A Posterior Agreement Approach

TL;DR

The paper addresses robustness of ML systems under covariate shift, arguing that accuracy alone is insufficient for evaluating resilience to distribution changes. It introduces Posterior Agreement (PA), a principled, supervision-free framework that compares two Gibbs posteriors, p(c|X′) and p(c|X″), under covariate perturbations by optimizing a shared inverse temperature β to maximize their overlap. The methodology provides a tractable, factorized posterior form and a robust kernel k(X′,X″) that captures agreement beyond task performance. Through extensive experiments on adversarial attacks and domain generalization benchmarks, PA demonstrates superior discriminability and stability compared to accuracy-based metrics and shows promise for robust model selection under realistic shift scenarios. The work lays groundwork for a principled robustness assessment that decouples performance from confidence and highlights paths for future theoretical and algorithmic extensions.

Abstract

The robustness of algorithms against covariate shifts is a fundamental problem with critical implications for the deployment of machine learning algorithms in the real world. Current evaluation methods predominantly measure robustness through the lens of standard generalization, relying on task performance measures like accuracy. This approach lacks a theoretical justification and underscores the need for a principled foundation of robustness assessment under distribution shifts. In this work, we set the desiderata for a robustness measure, and we propose a novel principled framework for the robustness assessment problem that directly follows the Posterior Agreement (PA) theory of model validation. Specifically, we extend the PA framework to the covariate shift setting and propose a measure for robustness evaluation. We assess the soundness of our measure in controlled environments and through an empirical robustness analysis in two different covariate shift scenarios: adversarial learning and domain generalization. We illustrate the suitability of PA by evaluating several models under different nature and magnitudes of shift, and proportion of affected observations. The results show that PA offers a reliable analysis of the vulnerabilities in learning algorithms across different shift conditions and provides higher discriminability than accuracy-based measures, while requiring no supervision.

Figures (18)

• Figure 1: Comparison of accuracy and our proposed measure, Posterior Agreement (PA), to assess the robustness of three binary classifiers. We simulate a dataset of targets $D_Y = \{y_1, \dots, y_n\}$, $|D_Y| = N$ where each $y_i \sim \mathcal{B}(p)$ is sampled from a Bernoulli distribution with $p = p(Y^\prime = 1)$ (displayed on the $x$-axis). We assume a perfect classifier $f(y_i) = y_i$, a constant classifier $f(y_i) = 0$, and a random classifier whose outputs are a permutation of $D_Y$, so that the number of mismatched observations depends on $p$. Accuracy does not comply with the desired properties of a robustness measure and provides an inconsistent assessment, which is exclusively driven by task performance. Instead, $\text{PA}$ detects the robustness of a constant classifier, discriminating it from the random, unrobust one.
• Figure 2: The $\text{PA}$ framework, applied in the estimation of a real-valued parameter (e.g., a distribution's mean). Two posteriors $p(c \mid {X}^{{\prime}})$, $p(c \mid {X}^{\prime\!\:\!\prime})$, $c \in \mathbb{R}$ are fit to two different data realizations. The optimization over the inverse temperature parameter $\beta$ is required to make the two posteriors insensitive to sampling noise. When $\beta$ is too low (top left), the posteriors tend to be uniform and uninformative, i.e., they underfit. When $\beta$ is too high (top right), the posteriors are too peaked and therefore sensitive to noise perturbation, i.e., they overfit. An optimal $\beta$ (top center) maximizes the posterior agreement (bottom), ensuring informativeness and stability that can be then used for model selection or, in our case, for robustness assessment. Illustration based on DBLP:journals/tcs/BuhmannDGS18.
• Figure 3: $\text{PA}$ (left) and $\text{AFR}_T$ (right) scores against increasing AR and $\ell_\infty$, for the PGD attack. The tendencies are similar with the only exception of the undefended model, which is overperforming according to $\text{AFR}_T$.
• Figure 4: (top) Distribution of the predictive confidence for three example models. The robust one DBLP:conf/icml/WangPDL0Y23 is significantly decreasing its confidence, while the weak ones (undefended and JPEG + RS) are not. (bottom) Final $\beta$ values for $AR = 1$. Anomalous $\beta$ values identify the weak models.
• Figure 5: $\text{PA}$ (left) and $\text{AFR}_T$ (right) scores against increasing AR and $\ell_\infty$, for the FMN attack. Again, the undefended model robustness is overestimated, according to $\text{AFR}_T$. For $AR \in [0.3, 0.6]$ JPEG + RS model is more robust than the others, with a similar trend in performance.

Theorems & Definitions (11)

• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3
• proof
• Lemma 1
• proof
• proof
• proof