Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TablePuppet: A Generic Framework for Relational Federated Learning

Lijie Xu, Chulin Xie, Yiran Guo, Gustavo Alonso, Bo Li, Guoliang Li, Wei Wang, Wentao Wu, Ce Zhang

TL;DR

This paper proposes TablePuppet, a generic framework for RFL that decomposes the learning process into two steps: learning over join (LoJ) followed by learning over union (LoU), and shows that TablePuppet achieves model accuracy comparable to the centralized baselines running directly atop the SQL results.

Abstract

Current federated learning (FL) approaches view decentralized training data as a single table, divided among participants either horizontally (by rows) or vertically (by columns). However, these approaches are inadequate for handling distributed relational tables across databases. This scenario requires intricate SQL operations like joins and unions to obtain the training data, which is either costly or restricted by privacy concerns. This raises the question: can we directly run FL on distributed relational tables? In this paper, we formalize this problem as relational federated learning (RFL). We propose TablePuppet, a generic framework for RFL that decomposes the learning process into two steps: (1) learning over join (LoJ) followed by (2) learning over union (LoU). In a nutshell, LoJ pushes learning down onto the vertical tables being joined, and LoU further pushes learning down onto the horizontal partitions of each vertical table. TablePuppet incorporates computation/communication optimizations to deal with the duplicate tuples introduced by joins, as well as differential privacy (DP) to protect against both feature and label leakages. We demonstrate the efficiency of TablePuppet in combination with two widely-used ML training algorithms, stochastic gradient descent (SGD) and alternating direction method of multipliers (ADMM), and compare their computation/communication complexity. We evaluate the SGD/ADMM algorithms developed atop TablePuppet by training diverse ML models. Our experimental results show that TablePuppet achieves model accuracy comparable to the centralized baselines running directly atop the SQL results. Moreover, ADMM takes less communication time than SGD to converge to similar model accuracy.

TablePuppet: A Generic Framework for Relational Federated Learning

TL;DR

This paper proposes TablePuppet, a generic framework for RFL that decomposes the learning process into two steps: learning over join (LoJ) followed by learning over union (LoU), and shows that TablePuppet achieves model accuracy comparable to the centralized baselines running directly atop the SQL results.

Abstract

Current federated learning (FL) approaches view decentralized training data as a single table, divided among participants either horizontally (by rows) or vertically (by columns). However, these approaches are inadequate for handling distributed relational tables across databases. This scenario requires intricate SQL operations like joins and unions to obtain the training data, which is either costly or restricted by privacy concerns. This raises the question: can we directly run FL on distributed relational tables? In this paper, we formalize this problem as relational federated learning (RFL). We propose TablePuppet, a generic framework for RFL that decomposes the learning process into two steps: (1) learning over join (LoJ) followed by (2) learning over union (LoU). In a nutshell, LoJ pushes learning down onto the vertical tables being joined, and LoU further pushes learning down onto the horizontal partitions of each vertical table. TablePuppet incorporates computation/communication optimizations to deal with the duplicate tuples introduced by joins, as well as differential privacy (DP) to protect against both feature and label leakages. We demonstrate the efficiency of TablePuppet in combination with two widely-used ML training algorithms, stochastic gradient descent (SGD) and alternating direction method of multipliers (ADMM), and compare their computation/communication complexity. We evaluate the SGD/ADMM algorithms developed atop TablePuppet by training diverse ML models. Our experimental results show that TablePuppet achieves model accuracy comparable to the centralized baselines running directly atop the SQL results. Moreover, ADMM takes less communication time than SGD to converge to similar model accuracy.
Paper Structure (36 sections, 2 theorems, 7 equations, 8 figures, 7 tables, 1 algorithm)

This paper contains 36 sections, 2 theorems, 7 equations, 8 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Relational Federated Learning
  3. Example Scenario of RFL
  4. Problem Formalization
  5. Limitations of Existing Work
  6. Overview of TablePuppet
  7. Learning over Join on Vertical Tables (LoJ)
  8. Table-Mapping Mechanism
  9. Problem Decomposition and Push Down
  10. Computation and Communication Reduction
  11. Learning over Union on Horizontal Tables
  12. Implementation of TablePuppet
  13. Training Process with Physical Operators
  14. SGD/ADMM with TablePuppet
  15. Privacy Guarantees
  16. ...and 21 more sections

Key Result

Theorem 1

(Privacy guarantee for labels.) Following standard Laplace mechanism (dwork2014algorithmic), $\epsilon$-label DP can be achieved by injecting additive Laplace noise with per-coordinate standard deviation $\lambda=\frac{2\sqrt{2}}{\epsilon}$.

Figures (8)

  • Figure 1: A RFL example, using UCQ (with SQL syntax) to express the joins and unions over three tables.
  • Figure 2: Illustration of RFL where the table $X$ with column $y$ is the UCQ result. Each horizontal table $T_i^q$ is owned by a client $c_i^q$.
  • Figure 3: TablePuppet framework and its training process. The red (black) locks indicate locations where DP is (can be) applied.
  • Figure 4: The architecture of TablePuppet.
  • Figure 5: The convergence rates of different algorithms for VFL/RFL-V and RFL scenarios without privacy guarantees.
  • ...and 3 more figures

Theorems & Definitions (3)

  • Definition 1: $(\epsilon,\delta)$-DP dwork2014algorithmic
  • Theorem 1
  • Theorem 2