A federated large language model for long-term time series forecasting

TL;DR

This work tackles privacy and scalability in long-term time-series forecasting by introducing FedTime, a federated large language model framework built on LLaMA-2 and augmented with channel independence, patch-based encoding, and two-phase fine-tuning. It leverages K-means clustering to partition edge devices, PEFT with QLoRA to minimize on-device updates, and DPO for alignment, achieving strong forecasting accuracy across Weather, Traffic, Electricity, and ETT datasets while reducing communication overhead. Empirical results show FedTime outperforms both centralized and federated baselines, with faster convergence and significant gains at longer horizons. The framework demonstrates practical impact for distributed IoT deployments by balancing privacy, efficiency, and predictive performance, with future directions including blockchain-enabled distributed systems.

Abstract

Long-term time series forecasting in centralized environments poses unique challenges regarding data privacy, communication overhead, and scalability. To address these challenges, we propose FedTime, a federated large language model (LLM) tailored for long-range time series prediction. Specifically, we introduce a federated pre-trained LLM with fine-tuning and alignment strategies. Prior to the learning process, we employ K-means clustering to partition edge devices or clients into distinct clusters, thereby facilitating more focused model training. We also incorporate channel independence and patching to better preserve local semantic information, ensuring that important contextual details are retained while minimizing the risk of information loss. We demonstrate the effectiveness of our FedTime model through extensive experiments on various real-world forecasting benchmarks, showcasing substantial improvements over recent approaches. In addition, we demonstrate the efficiency of FedTime in streamlining resource usage, resulting in reduced communication overhead.

Figures (6)

• Figure 1: FedTime Architecture. (a) Supervised fine-tuning with direct preference optimization (DPO) integration for backbone model alignment. (b) Forecasting fine-tuning using reversible instance normalization (RevIN).
• Figure 2: Long-term forecasting performance with varying look-back window lengths $L\in\{24, 48, 96, 192, 336, 720\}$. We set the prediction horizon to $T=720$.
• Figure 3: Training and testing learning curves of the centralized LLaMA model (left) and FedTime (right).
• Figure 4: ACN energy delivery distribution based on disconnection time.
• Figure 5: Communication overhead comparison between FedTime and baseline methods.