EviATTA: Evidential Active Test-Time Adaptation for Medical Segment Anything Models

Abstract

Deploying foundational medical Segment Anything Models (SAMs) via test-time adaptation (TTA) is challenging under large distribution shifts, where test-time supervision is often unreliable. While active test-time adaptation (ATTA) introduces limited expert feedback to improve reliability, existing ATTA methods still suffer from unreliable uncertainty estimation and inefficient utilization of sparse annotations. To address these issues, we propose Evidential Active Test-Time Adaptation (EviATTA), which is, to our knowledge, the first ATTA framework tailored for medical SAMs. Specifically, we adopt the Dirichlet-based Evidential Modeling to decompose overall predictive uncertainty into distribution uncertainty and data uncertainty. Building on this decomposition, we design a Hierarchical Evidential Sampling strategy, where image-wise distribution uncertainty is used to select informative shifted samples, while distance-aware data uncertainty guides sparse pixel annotations to resolve data ambiguities. We further introduce Dual Consistency Regularization, which enforces progressive prompt consistency on sparsely labeled samples to better exploit sparse supervision and applies variational feature consistency on unlabeled samples to stabilize adaptation. Extensive experiments on six medical image segmentation datasets demonstrate that EviATTA consistently improves adaptation reliability with minimal expert feedback under both batch-wise and instance-wise test-time adaptation settings.

Figures (8)

• Figure 1: Comparison of TTA, vanilla ATTA, and EviATTA. (a) TTA suffers from unreliable adaptation due to unreliable self-supervision. (b) Vanilla ATTA partially improves adaptation but still struggles with unreliable uncertainty estimation and inefficient utilization of sparse annotations. (c) EviATTA employs decoupled uncertainty for more reliable uncertainty estimation and leverages dual consistency regularization to exploit sparse annotations.
• Figure 2: Framework of EviATTA. It comprises three components: (a) Dirichlet-based Evidential Modeling, which formulates categorical prediction as following a Dirichlet distribution; (b) Hierarchical Evidential Sampling, which decomposes overall uncertainty into distribution and data uncertainties, selects samples via image-wise distribution uncertainty, and annotates pixels using pixel-wise data uncertainty; (c) Dual Consistency Regularization, which enforces progressive prompt consistency to leverage sparse annotations and variational feature consistency to stabilize adaptation.
• Figure 3: Visualization of uncertainty map. $U$ denotes predictive uncertainty. $U_{dis}$ and $U_{data}$ represent distribution and data uncertainties.
• Figure 4: Effect of pixel annotation budgets on BraTS-SSA.
• Figure 5: Effect of buffer size on BraTS-SSA.