Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One-Shot Price Forecasting with Covariate-Guided Experts under Privacy Constraints

Ren He, Yinliang Xu, Jinfeng Wang, Jeremy Watson, Jian Song

TL;DR

This paper tackles multivariate time-series forecasting under strict privacy constraints in power systems by introducing Cov-SMoE, a covariate-guided sparse mixture-of-experts module inserted between tokenization and encoder of pretrained transformer-based models. The MoE-Encoder enables two capabilities: transforming multivariate forecasting into a covariate-guided univariate core task and supporting federated, parameter-efficient personalization where raw data remain local. Empirical results on five public electricity datasets show consistent accuracy gains over strong baselines and robust generalization in non-IID, privacy-preserving federated settings, with substantial communication savings. The work offers a scalable, privacy-compatible extension to foundation time-series models, enabling region-specific adaptation without compromising data confidentiality and providing practical benefits for regulated domains.

Abstract

Forecasting in power systems often involves multivariate time series with complex dependencies and strict privacy constraints across regions. Traditional forecasting methods require significant expert knowledge and struggle to generalize across diverse deployment scenarios. Recent advancements in pre-trained time series models offer new opportunities, but their zero-shot performance on domain-specific tasks remains limited. To address these challenges, we propose a novel MoE Encoder module that augments pretrained forecasting models by injecting a sparse mixture-of-experts layer between tokenization and encoding. This design enables two key capabilities: (1) trans forming multivariate forecasting into an expert-guided univariate task, allowing the model to effectively capture inter-variable relations, and (2) supporting localized training and lightweight parameter sharing in federated settings where raw data cannot be exchanged. Extensive experiments on public multivariate datasets demonstrate that MoE-Encoder significantly improves forecasting accuracy compared to strong baselines. We further simulate federated environments and show that transferring only MoE-Encoder parameters allows efficient adaptation to new regions, with minimal performance degradation. Our findings suggest that MoE-Encoder provides a scalable and privacy-aware extension to foundation time series models.

One-Shot Price Forecasting with Covariate-Guided Experts under Privacy Constraints

TL;DR

This paper tackles multivariate time-series forecasting under strict privacy constraints in power systems by introducing Cov-SMoE, a covariate-guided sparse mixture-of-experts module inserted between tokenization and encoder of pretrained transformer-based models. The MoE-Encoder enables two capabilities: transforming multivariate forecasting into a covariate-guided univariate core task and supporting federated, parameter-efficient personalization where raw data remain local. Empirical results on five public electricity datasets show consistent accuracy gains over strong baselines and robust generalization in non-IID, privacy-preserving federated settings, with substantial communication savings. The work offers a scalable, privacy-compatible extension to foundation time-series models, enabling region-specific adaptation without compromising data confidentiality and providing practical benefits for regulated domains.

Abstract

Forecasting in power systems often involves multivariate time series with complex dependencies and strict privacy constraints across regions. Traditional forecasting methods require significant expert knowledge and struggle to generalize across diverse deployment scenarios. Recent advancements in pre-trained time series models offer new opportunities, but their zero-shot performance on domain-specific tasks remains limited. To address these challenges, we propose a novel MoE Encoder module that augments pretrained forecasting models by injecting a sparse mixture-of-experts layer between tokenization and encoding. This design enables two key capabilities: (1) trans forming multivariate forecasting into an expert-guided univariate task, allowing the model to effectively capture inter-variable relations, and (2) supporting localized training and lightweight parameter sharing in federated settings where raw data cannot be exchanged. Extensive experiments on public multivariate datasets demonstrate that MoE-Encoder significantly improves forecasting accuracy compared to strong baselines. We further simulate federated environments and show that transferring only MoE-Encoder parameters allows efficient adaptation to new regions, with minimal performance degradation. Our findings suggest that MoE-Encoder provides a scalable and privacy-aware extension to foundation time series models.
Paper Structure (24 sections, 1 equation, 1 figure, 4 tables, 2 algorithms)

This paper contains 24 sections, 1 equation, 1 figure, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Transformer-Based Time Series Forecasting
  4. Covariate-Aware Forecasting
  5. Federated Time Series Forecasting and Privacy
  6. Mixture-of-Experts for Structured Adaptation
  7. Transfer and Adaptation in Multivariate Forecasting
  8. Summary
  9. Method
  10. Overview
  11. Problem Formulation
  12. MoE-Encoder Architecture
  13. Input Tokenization
  14. Sparse Expert Routing
  15. Federated Compatibility
  16. ...and 9 more sections

Figures (1)

  • Figure 1: Covariate-aware Sparse Mixture-of-Experts (Cov-SMoE) module. It comprises three types of experts: Shared Experts (gray), which are always active; Conditional Experts (orange), selected based on external covariates (e.g., time, region); and Routed Experts (blue), selected via a token-level gating function. The Conditional Router uses auxiliary information to control deterministic expert assignment, while the Router computes top-$k$ scores over experts for input tokens. Selected expert outputs are aggregated and forwarded to the next layer.