Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Resilient Practical Test-Time Adaptation: Soft Batch Normalization Alignment and Entropy-driven Memory Bank

Xingzhi Zhou, Zhiliang Tian, Ka Chun Cheung, Simon See, Nevin L. Zhang

TL;DR

This paper tackles test-time domain adaptation under continual target-domain shifts and temporal correlations by proposing ResiTTA, a three-component framework. It combines Resilient Batch Normalization (ResiBN) to stabilize statistics with soft alignment, an Entropy-Driven Memory Bank (EntroBank) to curate high-quality memory samples, and a teacher-student self-training scheme to periodically adapt the model using memory data. ResiBN uses exponential updates of target statistics and $W_2^2$-based soft alignment between target and source BN statistics, while EntroBank manages timeliness, long-persisted over-confident samples, and sample uncertainty to maintain data quality. Empirical results on CIFAR10-C, CIFAR100-C, and ImageNet-C show state-of-the-art performance under practical test-time adaptation, with substantial improvements in average error over strong baselines and robustness across corruption types.

Abstract

Test-time domain adaptation effectively adjusts the source domain model to accommodate unseen domain shifts in a target domain during inference. However, the model performance can be significantly impaired by continuous distribution changes in the target domain and non-independent and identically distributed (non-i.i.d.) test samples often encountered in practical scenarios. While existing memory bank methodologies use memory to store samples and mitigate non-i.i.d. effects, they do not inherently prevent potential model degradation. To address this issue, we propose a resilient practical test-time adaptation (ResiTTA) method focused on parameter resilience and data quality. Specifically, we develop a resilient batch normalization with estimation on normalization statistics and soft alignments to mitigate overfitting and model degradation. We use an entropy-driven memory bank that accounts for timeliness, the persistence of over-confident samples, and sample uncertainty for high-quality data in adaptation. Our framework periodically adapts the source domain model using a teacher-student model through a self-training loss on the memory samples, incorporating soft alignment losses on batch normalization. We empirically validate ResiTTA across various benchmark datasets, demonstrating state-of-the-art performance.

Resilient Practical Test-Time Adaptation: Soft Batch Normalization Alignment and Entropy-driven Memory Bank

TL;DR

This paper tackles test-time domain adaptation under continual target-domain shifts and temporal correlations by proposing ResiTTA, a three-component framework. It combines Resilient Batch Normalization (ResiBN) to stabilize statistics with soft alignment, an Entropy-Driven Memory Bank (EntroBank) to curate high-quality memory samples, and a teacher-student self-training scheme to periodically adapt the model using memory data. ResiBN uses exponential updates of target statistics and -based soft alignment between target and source BN statistics, while EntroBank manages timeliness, long-persisted over-confident samples, and sample uncertainty to maintain data quality. Empirical results on CIFAR10-C, CIFAR100-C, and ImageNet-C show state-of-the-art performance under practical test-time adaptation, with substantial improvements in average error over strong baselines and robustness across corruption types.

Abstract

Test-time domain adaptation effectively adjusts the source domain model to accommodate unseen domain shifts in a target domain during inference. However, the model performance can be significantly impaired by continuous distribution changes in the target domain and non-independent and identically distributed (non-i.i.d.) test samples often encountered in practical scenarios. While existing memory bank methodologies use memory to store samples and mitigate non-i.i.d. effects, they do not inherently prevent potential model degradation. To address this issue, we propose a resilient practical test-time adaptation (ResiTTA) method focused on parameter resilience and data quality. Specifically, we develop a resilient batch normalization with estimation on normalization statistics and soft alignments to mitigate overfitting and model degradation. We use an entropy-driven memory bank that accounts for timeliness, the persistence of over-confident samples, and sample uncertainty for high-quality data in adaptation. Our framework periodically adapts the source domain model using a teacher-student model through a self-training loss on the memory samples, incorporating soft alignment losses on batch normalization. We empirically validate ResiTTA across various benchmark datasets, demonstrating state-of-the-art performance.
Paper Structure (26 sections, 19 equations, 1 figure, 10 tables, 1 algorithm)

This paper contains 26 sections, 19 equations, 1 figure, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Domain Adaptation
  4. Test-Time Domain Adaptation
  5. Method
  6. Problem Definition
  7. Resilient Practical Test Time Adaptation
  8. Resilient Batch Normalization
  9. Entropy-driven Memory Bank
  10. Self-Training Adapatation
  11. Experiments
  12. Datasets
  13. Baselines
  14. Implementation Details
  15. Main Results
  16. ...and 11 more sections

Figures (1)

  • Figure 1: Framework Overview. We duplicate the pre-trained model into a student and a teacher model at the beginning of the test and replace the batch normalization layer with resilient batch normalization (ResiBN). In the inference stage, we use the teacher model to predict labels. In the adaptation stage, we collect online test streams by entropy-drive memory bank (EntroBank). We periodically adapt the model through a self-training loss on data drawn from the memory bank, incorporating soft alignment losses.