Feature Augmentation based Test-Time Adaptation

TL;DR

Feature Augmentation based Test-time Adaptation (FATA) is proposed, a simple method that fully utilizes the limited amount of input data through feature augmentation.

Abstract

Test-time adaptation (TTA) allows a model to be adapted to an unseen domain without accessing the source data. Due to the nature of practical environments, TTA has a limited amount of data for adaptation. Recent TTA methods further restrict this by filtering input data for reliability, making the effective data size even smaller and limiting adaptation potential. To address this issue, We propose Feature Augmentation based Test-time Adaptation (FATA), a simple method that fully utilizes the limited amount of input data through feature augmentation. FATA employs Normalization Perturbation to augment features and adapts the model using the FATA loss, which makes the outputs of the augmented and original features similar. FATA is model-agnostic and can be seamlessly integrated into existing models without altering the model architecture. We demonstrate the effectiveness of FATA on various models and scenarios on ImageNet-C and Office-Home, validating its superiority in diverse real-world conditions.

Figures (5)

• Figure 1: Problem analysis. We use ImageNet pretrained ResNet-50 he2016deep and Gaussian noise of level 5 from ImageNet-C hendrycks2018benchmarking. We follow EATA niu2022efficient and SAR niu2022towards_SAR to set the entropy threshold. (a) 64.0% of the classes are selected 5 times or fewer, where each class contains 50 images. (b) The less frequently a class is selected, the lower the performance. (c) Only 11.85% of samples in ImageNet-C are used when entropy based filtering is used.
• Figure 2: Overview of FATA. There are two prediction branches where one is for obtaining pseudo-label on the reliable data and another is for prediction and updating the model on the augmented feature. We insert the feature augmentation after the $i$-th layer.
• Figure 3: Augmentation losses for the ablation study. $\mathbf{p}$ and $\mathbf{p'}$ denotes $\mathbf{p}_\theta (\mathbf{z})$ and $\mathbf{p}_\theta (\mathbf{z'})$, respectively.
• Figure 4: Hyperparameter Sensitivity.
• Figure 5: Count of classes for each number group of selected samples.