On Uncertainty Calibration for Equivariant Functions

TL;DR

This work develops a theoretical link between symmetry (equivariance) and model calibration, deriving upper and lower bounds on calibration errors for classification ($\text{ECE}$) and regression ($\text{ENCE}$, generalized as $GENCE$) under various equivariance conditions. It introduces the concepts of correct, incorrect, and extrinsic equivariance and the novel aleatoric bleed to quantify misattribution of uncertainty under symmetry constraints. Central to the theory is a fiber- and orbit-based decomposition using the fundamental domain, enabling precise bounds that depend on the ground-truth symmetry alignment and the dissent across orbits. Empirically, the authors show that symmetry mismatch typically worsens calibration and can induce aleatoric bleed, while correct invariance does not guarantee calibration gains, emphasizing the nuanced role of symmetry in uncertainty quantification across domains like Swiss Rolls, Galaxy Zoo morphologies, and chemical/vector-field tasks.

Abstract

Data-sparse settings such as robotic manipulation, molecular physics, and galaxy morphology classification are some of the hardest domains for deep learning. For these problems, equivariant networks can help improve modeling across undersampled parts of the input space, and uncertainty estimation can guard against overconfidence. However, until now, the relationships between equivariance and model confidence, and more generally equivariance and model calibration, has yet to be studied. Since traditional classification and regression error terms show up in the definitions of calibration error, it is natural to suspect that previous work can be used to help understand the relationship between equivariance and calibration error. In this work, we present a theory relating equivariance to uncertainty estimation. By proving lower and upper bounds on uncertainty calibration errors (ECE and ENCE) under various equivariance conditions, we elucidate the generalization limits of equivariant models and illustrate how symmetry mismatch can result in miscalibration in both classification and regression. We complement our theoretical framework with numerical experiments that clarify the relationship between equivariance and uncertainty using a variety of real and simulated datasets, and we comment on trends with symmetry mismatch, group size, and aleatoric and epistemic uncertainties.

Figures (13)

• Figure 1: Binary Classification on a Unit Square with Translation Invariance. Blue and green represent true labels.
• Figure 2: Dataset with pointwise incorrect invariance. The first two panels indicate that the model is invariant $C_2$ reflections over the $x$-axis. The presence of diagonal lines in the first panel depicts that the model has $2$ confidence fibers. The last panel indicates that the model has rotation invariance. Colors represent labels in all panels.
• Figure 3: The left plot shows test accuracy for the z-invariant network (blue) and baseline unconstrained MLP (red) under different ratios of correct/incorrect ratios, ranging from $0\%$ to $100\%$ correctness. The right hand plot is the same but for ECE instead of accuracy. As the correct ratio increases, the z-invariant MLP increases in test accuracy and decreases in test ECE, whereas the baseline MLP is relatively flat.
• Figure 4: Accuracy and ECE vs Cyclic Group Order under PSF Convolution on DESI Galaxies. The accuracy increases as a function of order especially with low PSF noise, however, the ECE varies tremendously.
• Figure 5: An example on how the ENCE upper bound behaves for an $E(2)-$invariant model $h$ on a set of point clouds and output space of vectors in $\mathbb{R}^n$, depicted as $1D-$lines. Each row shows an orbit from the representative set of $X$ in the first two columns as well as the lines produced by $f$ and $h_\mu$ in the third column, with the corresponding variance $s$ predicted by $h_{\sigma^2}(x)$ indicated by the ribbon. Each point cloud is titled with its name in the set $X$ and the probability of sampling it in $X$. It is noteworthy that the orbits in rows 1 and 2 are in the same confidence fiber, and only the second row has a nontrivial regression error lower bound for invariant models.

Theorems & Definitions (45)

• Definition 1: Definition 4.1 in wang2024general
• Definition 2: Point-wise Correct, Incorrect, and Extrinsic Equivariance, Definitions 3.5-3.7 in wang2024general
• Definition 3: Majority Label Total Dissent
• Theorem 1: Theorem 4.3 in wang2024general
• Theorem 2: Theorem 4.8 in wang2024general
• Theorem 3: Theorem 4.9 in wang2024general
• Proposition 1
• proof
• Theorem 4
• proof