Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FeDaL: Federated Dataset Learning for General Time Series Foundation Models

Shengchao Chen, Guodong Long, Michael Blumenstein, Jing Jiang

Abstract

Dataset-level heterogeneity introduces significant domain biases that fundamentally degrade generalization on general Time Series Foundation Models (TSFMs), yet this challenge remains underexplored. This paper rethinks the from-scratch training of TSFMs using the paradigm of federated learning. We propose a novel Federated Dataset Learning (FeDaL) approach to tackle heterogeneous time series by learning dataset-agnostic temporal representations. Specifically, the distributed architecture of federated learning is a nature solution to decompose heterogeneous TS datasets into shared generalized knowledge and preserved personalized knowledge. Moreover, based on the TSFM architecture, FeDaL explicitly mitigates both local and global biases by adding two complementary mechanisms: Domain Bias Elimination (DBE) and Global Bias Elimination (GBE). FeDaL`s cross-dataset generalization has been extensively evaluated in real-world datasets spanning eight tasks (including various regression and classification), against 54 baselines. We further analyze federated scaling behavior, showing how data volume, client count, and join rate affect model performance under decentralization. Our code is publicly available at https://github.com/shengchaochen82/FeDaL

FeDaL: Federated Dataset Learning for General Time Series Foundation Models

Abstract

Dataset-level heterogeneity introduces significant domain biases that fundamentally degrade generalization on general Time Series Foundation Models (TSFMs), yet this challenge remains underexplored. This paper rethinks the from-scratch training of TSFMs using the paradigm of federated learning. We propose a novel Federated Dataset Learning (FeDaL) approach to tackle heterogeneous time series by learning dataset-agnostic temporal representations. Specifically, the distributed architecture of federated learning is a nature solution to decompose heterogeneous TS datasets into shared generalized knowledge and preserved personalized knowledge. Moreover, based on the TSFM architecture, FeDaL explicitly mitigates both local and global biases by adding two complementary mechanisms: Domain Bias Elimination (DBE) and Global Bias Elimination (GBE). FeDaL`s cross-dataset generalization has been extensively evaluated in real-world datasets spanning eight tasks (including various regression and classification), against 54 baselines. We further analyze federated scaling behavior, showing how data volume, client count, and join rate affect model performance under decentralization. Our code is publicly available at https://github.com/shengchaochen82/FeDaL

Paper Structure

This paper contains 50 sections, 19 equations, 11 figures, 30 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Pre-trained Time Series Models.
  4. Federated Foundation Models.
  5. Methodology
  6. Domain Bias Elimination
  7. Global Bias Elimination
  8. Gradient-level Dynamic Correction.
  9. Core-set Tuning
  10. Main Results
  11. Federated Representation Learning
  12. Setup.
  13. Main Results.
  14. Bias Mitigation.
  15. Ablation & Hyperparameter Sensitivity Results.
  16. ...and 35 more sections

Figures (11)

  • Figure 1: Dataset-level biases. Information Lost: Low-resolution sequences capture fewer details under a fixed window. Pattern Break: Abrupt structural changes across time. Settings: FedAvg-Independent (each dataset as a client), FedAvg-Mixed (datasets pooled then split), and Stand-alone (no aggregation). 'Transformer' means that results from the local Transformer model. Setup follow Sec. 4. Lower MSE is better.
  • Figure 2: FeDaL: DBE reduces dataset-induced biases on the client side, whereas GBE improves alignment on the server. Locally, the plug-and-play DBE block (instead of DBE mechanism) captures trend and seasonal biases from latent representations.
  • Figure 3: Local bias change across rounds (R1 - R60) for select clients (Clients 3, 7, 9).
  • Figure 4: Hyperparameter sensitivity. Results on UTSD are averaged over H1 and H2. $\downarrow$: performance drop; $\uparrow$: improvement relative to the original FeDaL. Best viewed in color and with zoom.
  • Figure 5: Classification results. Full results in \ref{['tab:classification_ucr_full']} & \ref{['tab:classification_uea_full']}. Best viewed in color and with zoom.
  • ...and 6 more figures