Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedASTA: Federated adaptive spatial-temporal attention for traffic flow prediction

Kaiyuan Li, Yihan Zhang, Huandong Wang, Yan Zhuo, Xinlei Chen

TL;DR

A novel Federated Adaptive Spatial-Temporal Attention (FedASTA) framework to model the dynamic spatial-temporal relations and the effectiveness of the novel adaptive graph construction approach compared with other popular dynamic spatial-temporal aware methods is shown.

Abstract

Mobile devices and the Internet of Things (IoT) devices nowadays generate a large amount of heterogeneous spatial-temporal data. It remains a challenging problem to model the spatial-temporal dynamics under privacy concern. Federated learning (FL) has been proposed as a framework to enable model training across distributed devices without sharing original data which reduce privacy concern. Personalized federated learning (PFL) methods further address data heterogenous problem. However, these methods don't consider natural spatial relations among nodes. For the sake of modeling spatial relations, Graph Neural Netowork (GNN) based FL approach have been proposed. But dynamic spatial-temporal relations among edge nodes are not taken into account. Several approaches model spatial-temporal dynamics in a centralized environment, while less effort has been made under federated setting. To overcome these challeges, we propose a novel Federated Adaptive Spatial-Temporal Attention (FedASTA) framework to model the dynamic spatial-temporal relations. On the client node, FedASTA extracts temporal relations and trend patterns from the decomposed terms of original time series. Then, on the server node, FedASTA utilize trend patterns from clients to construct adaptive temporal-spatial aware graph which captures dynamic correlation between clients. Besides, we design a masked spatial attention module with both static graph and constructed adaptive graph to model spatial dependencies among clients. Extensive experiments on five real-world public traffic flow datasets demonstrate that our method achieves state-of-art performance in federated scenario. In addition, the experiments made in centralized setting show the effectiveness of our novel adaptive graph construction approach compared with other popular dynamic spatial-temporal aware methods.

FedASTA: Federated adaptive spatial-temporal attention for traffic flow prediction

TL;DR

A novel Federated Adaptive Spatial-Temporal Attention (FedASTA) framework to model the dynamic spatial-temporal relations and the effectiveness of the novel adaptive graph construction approach compared with other popular dynamic spatial-temporal aware methods is shown.

Abstract

Mobile devices and the Internet of Things (IoT) devices nowadays generate a large amount of heterogeneous spatial-temporal data. It remains a challenging problem to model the spatial-temporal dynamics under privacy concern. Federated learning (FL) has been proposed as a framework to enable model training across distributed devices without sharing original data which reduce privacy concern. Personalized federated learning (PFL) methods further address data heterogenous problem. However, these methods don't consider natural spatial relations among nodes. For the sake of modeling spatial relations, Graph Neural Netowork (GNN) based FL approach have been proposed. But dynamic spatial-temporal relations among edge nodes are not taken into account. Several approaches model spatial-temporal dynamics in a centralized environment, while less effort has been made under federated setting. To overcome these challeges, we propose a novel Federated Adaptive Spatial-Temporal Attention (FedASTA) framework to model the dynamic spatial-temporal relations. On the client node, FedASTA extracts temporal relations and trend patterns from the decomposed terms of original time series. Then, on the server node, FedASTA utilize trend patterns from clients to construct adaptive temporal-spatial aware graph which captures dynamic correlation between clients. Besides, we design a masked spatial attention module with both static graph and constructed adaptive graph to model spatial dependencies among clients. Extensive experiments on five real-world public traffic flow datasets demonstrate that our method achieves state-of-art performance in federated scenario. In addition, the experiments made in centralized setting show the effectiveness of our novel adaptive graph construction approach compared with other popular dynamic spatial-temporal aware methods.
Paper Structure (21 sections, 1 theorem, 12 equations, 5 figures, 5 tables)

This paper contains 21 sections, 1 theorem, 12 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Federated Learning
  4. Dynamic construction of graph
  5. Methodology
  6. Problem Formulation
  7. Client Model
  8. Adaptive Spatial-Temporal Graph Construction
  9. Modeling the Spatiotemporal Relations on Server
  10. Privacy Analysis
  11. Attack Model
  12. Protect Method
  13. Experiment
  14. Experiment Settings
  15. Results and Analyses
  16. ...and 6 more sections

Key Result

Theorem 1

A Cauchy mechanism with scale factor $c>\frac{\Delta f(1+e^{\epsilon/2})}{(e^\epsilon-1)(\frac{1}{2}+\tan(\pi\delta))}$ provides $(\epsilon,\delta)$-DP.

Figures (5)

  • Figure 1: (a) Different regions exhibit distinct temporal features. (b) Sensors with similar temporal features are not necessarily spatially adjacent.
  • Figure 2: Overall Structure
  • Figure 3: Performance of the model under different numbers of nodes and edges (left, RMSE; right, MAE)
  • Figure 4: Effect of privacy intensities.
  • Figure 5: Effectiveness of dynamic graph construction mechanism. (a)-(d) shows the attention maps of 4 disjoint periods. The darker color represents stronger correlations. The blue line and green line represent the similarity vector of nodes #30 and #225 respectively. (e)-(h) and (i)-(l) shows the different periods from 4 neighbors of nodes #30 and #225 and themselves respectively.

Theorems & Definitions (2)

  • Theorem 1: Constraints of Cauchy mechanism
  • proof