Conformal Uncertainty Indicator for Continual Test-Time Adaptation

TL;DR

This work tackles the problem of error accumulation in Continual Test-Time Adaptation (CTTA) caused by unreliable pseudo-labels. It introduces Conformal Uncertainty Indicator (CUI), a plug-and-play CP-based uncertainty estimator that generates prediction sets with coverage $1-\alpha$ and compensates for coverage gaps arising from continual domain shifts by jointly measuring model and data differences, producing a domain-shift-adjusted threshold $\hat{\tau}$. CUI also guides adaptation by weighting samples according to the reliability of their CP-based predictions, via CP Ada, improving robustness across CTTA methods on benchmarks such as CIFAR10C, CIFAR100C, and ImageNetC. The experimental results demonstrate that CUI yields reliable uncertainty estimates and enhances adaptation performance, while revealing tradeoffs related to calibration data storage and computation, making it suitable for safety-critical deployment with long-term domain drift.

Abstract

Continual Test-Time Adaptation (CTTA) aims to adapt models to sequentially changing domains during testing, relying on pseudo-labels for self-adaptation. However, incorrect pseudo-labels can accumulate, leading to performance degradation. To address this, we propose a Conformal Uncertainty Indicator (CUI) for CTTA, leveraging Conformal Prediction (CP) to generate prediction sets that include the true label with a specified coverage probability. Since domain shifts can lower the coverage than expected, making CP unreliable, we dynamically compensate for the coverage by measuring both domain and data differences. Reliable pseudo-labels from CP are then selectively utilized to enhance adaptation. Experiments confirm that CUI effectively estimates uncertainty and improves adaptation performance across various existing CTTA methods.

Figures (4)

• Figure 1: In the task of CTTA, a test sample $x$ may be drawn from a different distribution in a long-term testing phase. Traditional methods rely on the self-adaptation based on the prediction and ignore the uncertainty may cause error accumulation. CUI provides a technique of uncertainty measurement based on CP. For the test sample, if CUI outputs a prediction set with small sizes ($>0$), it is regarded as reliable and yields a large loss weight in adaptation. Large prediction sets mean unreliable prediction. The coverage means that the true label is included in the prediction set. The example image is sampled from ImageNet deng2009imagenet.
• Figure 2: Visualization of coverage and inefficiency changes.
• Figure 3: Hyperparameter analysis on CIFAR100-to-CIFAR100C.
• Figure 4: Time and memory cost on CIFAR100-to-CIFAR100C.