Augmented Contrastive Clustering with Uncertainty-Aware Prototyping for Time Series Test Time Adaptation

TL;DR

Augmented Contrastive Clustering with Uncertainty-aware Prototyping (ACCUP) is proposed, a straightforward yet effective TTA method for time series data that advances the underexplored realm of TTA for time series data.

Abstract

Test-time adaptation aims to adapt pre-trained deep neural networks using solely online unlabelled test data during inference. Although TTA has shown promise in visual applications, its potential in time series contexts remains largely unexplored. Existing TTA methods, originally designed for visual tasks, may not effectively handle the complex temporal dynamics of real-world time series data, resulting in suboptimal adaptation performance. To address this gap, we propose Augmented Contrastive Clustering with Uncertainty-aware Prototyping (ACCUP), a straightforward yet effective TTA method for time series data. Initially, our approach employs augmentation ensemble on the time series data to capture diverse temporal information and variations, incorporating uncertainty-aware prototypes to distill essential characteristics. Additionally, we introduce an entropy comparison scheme to selectively acquire more confident predictions, enhancing the reliability of pseudo labels. Furthermore, we utilize augmented contrastive clustering to enhance feature discriminability and mitigate error accumulation from noisy pseudo labels, promoting cohesive clustering within the same class while facilitating clear separation between different classes. Extensive experiments conducted on three real-world time series datasets and an additional visual dataset demonstrate the effectiveness and generalization potential of the proposed method, advancing the underexplored realm of TTA for time series data.

Figures (6)

• Figure 1: Illustration of SFDA and TTA for time series applications. (a) SFDA utilizes all available target data for multi-epoch adaptation before making final predictions. (b) TTA adapts the pre-trained model to target data batches in an online manner, where each batch can only be observed once.
• Figure 2: Illustration of the proposed model. Given the input signal, initial enhanced features and predictions are obtained via augmentation-ensemble, and then the predictions are further corrected by uncertainty-aware prototypes. Further, trustworthy pseudo-labels are obtained using an entropy comparison scheme. Lastly, more discriminative features are learnt using augmented contrastive clustering constraints to mitigate error accumulation and improve model efficacy.
• Figure 3: Illustration of the uncertainty-aware prototypes. The memorized support set is first initialized using the weight parameters of the linear classifier from the pre-trained source model. When the target domain batch of data arrives, the enhanced features and logits obtained from the augmentation-ensemble are deposited into the memorized support set, and then entropy filtering is used to select plausible representatives of the support set to produce a more reliable class prototypes.
• Figure 4: Analysis of adaptation performance with varying parameters. (a) $K$ for the memorized support set selection. (b) $\eta$ for uncertainty-aware prototypes component. (c) $\tau$ for augmented contrastive clustering component.
• Figure 5: t-SNE visualization of extracted feature distributions. Colors denote different categories. (a) depicts the source pre-trained model without adaptation on the UCIHAR dataset, (b) MFD dataset, and (c) SSC dataset. Following adaptation with the proposed model, (d) shows the MFD dataset, (e) UCIHAR dataset, and (f) SSC dataset.