Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heterogeneous Federated Learning Systems for Time-Series Power Consumption Prediction with Multi-Head Embedding Mechanism

Jia-Hao Syu, Jerry Chun-Wei Lin, Gautam Srivastava, Unil Yun

TL;DR

The paper tackles privacy-aware time-series power consumption prediction across heterogeneous clients by introducing Multi-Head Heterogeneous Federated Learning (MHHFL). It couples a data-preprocessing pipeline with multiple head networks, each producing a 2D output that is embedded and shared to a central source pool for asynchronous knowledge transfer via head selection and blending. Empirical results on AIUT and Husky datasets show substantial reductions in prediction error (up to $94.1\%$ versus SOTA) and robust performance under simulated latency, with ablations validating the importance of head embedding and selection mechanisms. The approach reduces privacy leakage by sharing only head-network information and offers security benefits through filtering and blending, making it practically impactful for privacy-preserving, multi-domain power prediction.

Abstract

Time-series prediction is increasingly popular in a variety of applications, such as smart factories and smart transportation. Researchers have used various techniques to predict power consumption, but existing models lack discussion of collaborative learning and privacy issues among multiple clients. To address these issues, we propose Multi-Head Heterogeneous Federated Learning (MHHFL) systems that consist of multiple head networks, which independently act as carriers for federated learning. In the federated period, each head network is embedded into 2-dimensional vectors and shared with the centralized source pool. MHHFL then selects appropriate source networks and blends the head networks as knowledge transfer in federated learning. The experimental results show that the proposed MHHFL systems significantly outperform the benchmark and state-of-the-art systems and reduce the prediction error by 24.9% to 94.1%. The ablation studies demonstrate the effectiveness of the proposed mechanisms in the MHHFL (head network embedding and selection mechanisms), which significantly outperforms traditional federated average and random transfer.

Heterogeneous Federated Learning Systems for Time-Series Power Consumption Prediction with Multi-Head Embedding Mechanism

TL;DR

The paper tackles privacy-aware time-series power consumption prediction across heterogeneous clients by introducing Multi-Head Heterogeneous Federated Learning (MHHFL). It couples a data-preprocessing pipeline with multiple head networks, each producing a 2D output that is embedded and shared to a central source pool for asynchronous knowledge transfer via head selection and blending. Empirical results on AIUT and Husky datasets show substantial reductions in prediction error (up to versus SOTA) and robust performance under simulated latency, with ablations validating the importance of head embedding and selection mechanisms. The approach reduces privacy leakage by sharing only head-network information and offers security benefits through filtering and blending, making it practically impactful for privacy-preserving, multi-domain power prediction.

Abstract

Time-series prediction is increasingly popular in a variety of applications, such as smart factories and smart transportation. Researchers have used various techniques to predict power consumption, but existing models lack discussion of collaborative learning and privacy issues among multiple clients. To address these issues, we propose Multi-Head Heterogeneous Federated Learning (MHHFL) systems that consist of multiple head networks, which independently act as carriers for federated learning. In the federated period, each head network is embedded into 2-dimensional vectors and shared with the centralized source pool. MHHFL then selects appropriate source networks and blends the head networks as knowledge transfer in federated learning. The experimental results show that the proposed MHHFL systems significantly outperform the benchmark and state-of-the-art systems and reduce the prediction error by 24.9% to 94.1%. The ablation studies demonstrate the effectiveness of the proposed mechanisms in the MHHFL (head network embedding and selection mechanisms), which significantly outperforms traditional federated average and random transfer.
Paper Structure (18 sections, 15 equations, 5 figures, 4 tables, 1 algorithm)

This paper contains 18 sections, 15 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Federated Learning
  4. Power Consumption Prediction
  5. Proposed Multi-Head Heterogeneous Federated Learning (MHHFL)
  6. Data Preprocessing
  7. Network Design
  8. Heterogeneous Federated Learning
  9. Head Network Embedding
  10. Selection and Blending Mechanisms
  11. MHHFL Overview
  12. Experimental Results
  13. Data Usage and Benchmark Systems
  14. Prediction Evaluation
  15. Robustness Evaluation
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Network design of MHHFL
  • Figure 2: Proposed heterogeneous federated learning
  • Figure 3: Robustness Evaluation
  • Figure 4: Ablation studies on four datasets
  • Figure 5: Sensitivity analysis on blending scale $\alpha$