Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement

TL;DR

This work tackles the challenge of efficient Source-Free Domain Adaptation for time-series by reparameterizing the source backbone with Tucker-style low-rank factorization and then selectively fine-tuning a compact core tensor on the target domain. The approach yields substantial parameter and MACs reductions while remaining compatible with multiple SFDA methods, and is supported by a PAC-Bayesian generalization framework that explains the implicit regularization during adaptation. Empirically, the method achieves robust sample- and parameter-efficiency across AdaTime benchmarks, including low-data regimes, and demonstrates superior performance compared to baseline SFDA methods and other parameter-efficient tuning approaches. The research highlights practical benefits for resource-constrained deployments and provides a principled, transferable framework for efficient domain adaptation in time-series settings.

Abstract

In this paper, we propose a framework for efficient Source-Free Domain Adaptation (SFDA) in the context of time-series, focusing on enhancing both parameter efficiency and data-sample utilization. Our approach introduces an improved paradigm for source-model preparation and target-side adaptation, aiming to enhance training efficiency during target adaptation. Specifically, we reparameterize the source model's weights in a Tucker-style decomposed manner, factorizing the model into a compact form during the source model preparation phase. During target-side adaptation, only a subset of these decomposed factors is fine-tuned, leading to significant improvements in training efficiency. We demonstrate using PAC Bayesian analysis that this selective fine-tuning strategy implicitly regularizes the adaptation process by constraining the model's learning capacity. Furthermore, this re-parameterization reduces the overall model size and enhances inference efficiency, making the approach particularly well suited for resource-constrained devices. Additionally, we demonstrate that our framework is compatible with various SFDA methods and achieves significant computational efficiency, reducing the number of fine-tuned parameters and inference overhead in terms of MACs by over 90% while maintaining model performance.

Figures (20)

• Figure 1: Prior Paradigm vs. Ours.A. Existing approaches SHOTNRCAaDMAPU adapt the entire model to the target distribution. B. Our method disentangles the backbone parameters using Tucker-style factorization. Fine-tuning a small subset of these parameters proves both parameter- and sample-efficient (cf. Section \ref{['sec:efficiency']}), while also being robust against overfitting (cf. Figure \ref{['fig:overfitting']}B and Section \ref{['sec:robust_adaptation']}), leading to superior adaptation to the target distribution.
• Figure 2: A. Original Dimensions vs. Effective Rank: Comparison of input/output channel dimensions ($C_{\text{in}}, C_{\text{out}}$) in the last two layers of the source model trained on the HHAR HHAR and SSC SleepEDF datasets, alongside their effective rank computed via VBMF nakajima13a. B. Regularization Effect: Training dynamics illustrate the regularization effect of our source decomposition and selective fine-tuning (SFT) compared to the SHOT (Baseline) SHOT on the SSC dataset.
• Figure 3: Illustration of Tucker-style factorization Tuck1966cLathauwerMult2000.
• Figure 4: The two columns show the inputs from MNIST-1D Greydanus2020ScalingDD and the core tensors for the source and target domains. The target domain is generated manually by vertically flipping (i.e., negating) the source. Both $R_{\text{in}}$ and $R_{\text{out}}$ are set to 1 (cf. Figure \ref{['fig:tucker']}B) to facilitate visualization of the core tensors. Domain-invariant output features are achieved as the core tensor adapts to the target samples to mitigate the domain shift (negation).
• Figure 5: Layer-wise parameter distance between the source-pretrained model and the target-adapted model using the SFT strategy for different rank-factor values ($RF \in \{2, 4, 8\}$) on the SSC SleepEDF, MFD MFD, and HHAR HHAR datasets. The values represent the average parameter distances across all source-target pairs provided for the respective datasets. Lower values indicate smaller parameter distances.

Theorems & Definitions (4)

• proof
• proof
• proof
• proof