Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PiXTime: A Model for Federated Time Series Forecasting with Heterogeneous Data Structures Across Nodes

Yiming Zhou, Mingyue Cheng, Hao Wang, Enhong Chen

TL;DR

The paper tackles federated time series forecasting when nodes exhibit heterogeneous sampling rates and variable sets. It introduces PiXTime, a transformer-based model featuring a node-specific Patch Embedding that maps target sequences of length $T_i$ into a uniform-dimensional token space of dimension $D$ using $M_i=T_i/PL_i$ patches, and a global VE Table that maintains consistent embeddings for each variable category. The architecture includes an Auxiliary Encoder to learn variable-wise representations and a Target Decoder that uses cross-attention via an abstract token to fuse auxiliary information into patch-level target representations, with a local Projection Head producing forecasts $x_i^{pre}$; shared modules are aggregated under FedOPT while Patch Embedding remains node-local. Empirical results on eight real-world benchmarks show PiXTime achieves state-of-the-art performance in federated settings and strong non-federated results, with ablations confirming the critical roles of Patch Embedding and VE Table in handling heterogeneity.

Abstract

Time series are highly valuable and rarely shareable across nodes, making federated learning a promising paradigm to leverage distributed temporal data. However, different sampling standards lead to diverse time granularities and variable sets across nodes, hindering classical federated learning. We propose PiXTime, a novel time series forecasting model designed for federated learning that enables effective prediction across nodes with multi-granularity and heterogeneous variable sets. PiXTime employs a personalized Patch Embedding to map node-specific granularity time series into token sequences of a unified dimension for processing by a subsequent shared model, and uses a global VE Table to align variable category semantics across nodes, thereby enhancing cross-node transferability. With a transformer-based shared model, PiXTime captures representations of auxiliary series with arbitrary numbers of variables and uses cross-attention to enhance the prediction of the target series. Experiments show PiXTime achieves state-of-the-art performance in federated settings and demonstrates superior performance on eight widely used real-world traditional benchmarks.

PiXTime: A Model for Federated Time Series Forecasting with Heterogeneous Data Structures Across Nodes

TL;DR

The paper tackles federated time series forecasting when nodes exhibit heterogeneous sampling rates and variable sets. It introduces PiXTime, a transformer-based model featuring a node-specific Patch Embedding that maps target sequences of length into a uniform-dimensional token space of dimension using patches, and a global VE Table that maintains consistent embeddings for each variable category. The architecture includes an Auxiliary Encoder to learn variable-wise representations and a Target Decoder that uses cross-attention via an abstract token to fuse auxiliary information into patch-level target representations, with a local Projection Head producing forecasts ; shared modules are aggregated under FedOPT while Patch Embedding remains node-local. Empirical results on eight real-world benchmarks show PiXTime achieves state-of-the-art performance in federated settings and strong non-federated results, with ablations confirming the critical roles of Patch Embedding and VE Table in handling heterogeneity.

Abstract

Time series are highly valuable and rarely shareable across nodes, making federated learning a promising paradigm to leverage distributed temporal data. However, different sampling standards lead to diverse time granularities and variable sets across nodes, hindering classical federated learning. We propose PiXTime, a novel time series forecasting model designed for federated learning that enables effective prediction across nodes with multi-granularity and heterogeneous variable sets. PiXTime employs a personalized Patch Embedding to map node-specific granularity time series into token sequences of a unified dimension for processing by a subsequent shared model, and uses a global VE Table to align variable category semantics across nodes, thereby enhancing cross-node transferability. With a transformer-based shared model, PiXTime captures representations of auxiliary series with arbitrary numbers of variables and uses cross-attention to enhance the prediction of the target series. Experiments show PiXTime achieves state-of-the-art performance in federated settings and demonstrates superior performance on eight widely used real-world traditional benchmarks.
Paper Structure (8 sections, 7 equations, 3 figures, 3 tables)

This paper contains 8 sections, 7 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. PiXTime
  4. Experiments
  5. Non-Federated Forecasting Results
  6. Federated Forecasting Results
  7. Effectiveness of Core Components in PiXTime
  8. Conclusion

Figures (3)

  • Figure 1: The dilemma of federated learning under diverse data structures. (a): Nodes with heterogeneous data structures collaboratively train a homogeneous model via federated learning. (b): The homogeneous model can only process data of a specific structure.
  • Figure 2: The architecture of PiXTime. (a) The target series is sliced temporally and mapped into a token sequence, then an abstract token is concatenated for subsequent processing. (b) Multiple auxiliary series are mapped variable-wise into a token sequence, then corresponding variable embeddings are added before further processing. (c) Representations from auxiliary series are transferred to the abstract token through cross-attention. (d) The blued modules of PiXTime are shared by federated learning, while the rest are kept local to each node.
  • Figure 3: Ablation study of the VE Table in federated networks with heterogeneous auxiliary variable sets across nodes.