Towards Reliable Test-Time Adaptation: Style Invariance as a Correctness Likelihood

TL;DR

Test-time adaptation often yields poorly calibrated uncertainty under dynamic deployment conditions. We introduce SICL, a plug-and-play calibration framework that uses style invariance to estimate instance-wise correctness likelihood by evaluating prediction consistency across style-shifted variants, without backpropagation. By perturbing style statistics and applying a content-based relaxation to prevent collapse, SICL delivers robust uncertainty estimates and improves calibration by about 13 percentage points on average across CIFAR-10/100-C and ImageNet-C under diverse TTA baselines. The approach achieves state-of-the-art calibration in most settings, including dynamic test streams, and offers strong practical impact for reliable deployment in high-stakes domains.

Abstract

Test-time adaptation (TTA) enables efficient adaptation of deployed models, yet it often leads to poorly calibrated predictive uncertainty - a critical issue in high-stakes domains such as autonomous driving, finance, and healthcare. Existing calibration methods typically assume fixed models or static distributions, resulting in degraded performance under real-world, dynamic test conditions. To address these challenges, we introduce Style Invariance as a Correctness Likelihood (SICL), a framework that leverages style-invariance for robust uncertainty estimation. SICL estimates instance-wise correctness likelihood by measuring prediction consistency across style-altered variants, requiring only the model's forward pass. This makes it a plug-and-play, backpropagation-free calibration module compatible with any TTA method. Comprehensive evaluations across four baselines, five TTA methods, and two realistic scenarios with three model architecture demonstrate that SICL reduces calibration error by an average of 13 percentage points compared to conventional calibration approaches.

Figures (9)

• Figure 1: Conceptual illustration of SICL's idea. SICL estimates correctness likelihood by assessing style invariance. Consistent predictions across style variants (generated by feature perturbation) (bottom) indicate high confidence in correctness, while low style invariance (top) indicate high uncertainty and lower confidence.
• Figure 2: Observations upon TTA calibration patterns: (a) & (b) show reliability and calibration error changes across temporal distribution shifts, while (c) & (d) display these changes under a fixed Gaussian Noise corruption.
• Figure 3: Motivative analysis for desirable ensemble candidates. From left to right, each visualize (a) calibration performance comparison between style and content perturbation, and (b) mean ContentVariance of ensemble candidates over various corruption types.
• Figure 4: Detailed illustration of the SICL framework, demonstrating how style invariance is leveraged to estimate correctness likelihood. Given an original sample $\mathbf{x}_i$, SICL generates multiple style-shifted variants as ensemble candidates through feature-level style perturbations. Prediction consistency across these variants is used as a final prediction confidence, where high style-invariance indicates a higher confidence.
• Figure 5: Qualitative analysis for style variants. From left to right, each visualize (a) mean ContentVariance over various corruption types, and (b) mean StyleVariance over various corruption types.