Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture

Yi Liu, Yang Liu, Leqian Zheng, Jue Hong, Junjie Shi, Qingyou Yang, Ye Wu, Cong Wang

TL;DR

PubSub-VFL presents a two-party vertical federated learning framework that decouples data ID alignment from training via a Publisher/Subscriber architecture and a hierarchical asynchronous mechanism. By combining Pub/Sub decoupling with parameter-server data parallelism, and optimizing hyperparameters through a privacy-preserving dynamic programming planner, it significantly reduces training latency while improving resource utilization. The approach includes GDP-based privacy for embeddings, buffer and deadline mechanisms to handle channel congestion, and a formal convergence analysis, validated by experiments on five benchmarks showing substantial speedups and robust performance under resource/data heterogeneity. This design offers practical advantages for privacy-preserving cross-institution learning in heterogeneous environments, with clear paths to extending to multi-party settings in future work.

Abstract

With the rapid advancement of the digital economy, data collaboration between organizations has become a well-established business model, driving the growth of various industries. However, privacy concerns make direct data sharing impractical. To address this, Two-Party Split Learning (a.k.a. Vertical Federated Learning (VFL)) has emerged as a promising solution for secure collaborative learning. Despite its advantages, this architecture still suffers from low computational resource utilization and training efficiency. Specifically, its synchronous dependency design increases training latency, while resource and data heterogeneity among participants further hinder efficient computation. To overcome these challenges, we propose PubSub-VFL, a novel VFL paradigm with a Publisher/Subscriber architecture optimized for two-party collaborative learning with high computational efficiency. PubSub-VFL leverages the decoupling capabilities of the Pub/Sub architecture and the data parallelism of the parameter server architecture to design a hierarchical asynchronous mechanism, reducing training latency and improving system efficiency. Additionally, to mitigate the training imbalance caused by resource and data heterogeneity, we formalize an optimization problem based on participants' system profiles, enabling the selection of optimal hyperparameters while preserving privacy. We conduct a theoretical analysis to demonstrate that PubSub-VFL achieves stable convergence and is compatible with security protocols such as differential privacy. Extensive case studies on five benchmark datasets further validate its effectiveness, showing that, compared to state-of-the-art baselines, PubSub-VFL not only accelerates training by $2 \sim 7\times$ without compromising accuracy, but also achieves a computational resource utilization rate of up to 91.07%.

PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture

TL;DR

PubSub-VFL presents a two-party vertical federated learning framework that decouples data ID alignment from training via a Publisher/Subscriber architecture and a hierarchical asynchronous mechanism. By combining Pub/Sub decoupling with parameter-server data parallelism, and optimizing hyperparameters through a privacy-preserving dynamic programming planner, it significantly reduces training latency while improving resource utilization. The approach includes GDP-based privacy for embeddings, buffer and deadline mechanisms to handle channel congestion, and a formal convergence analysis, validated by experiments on five benchmarks showing substantial speedups and robust performance under resource/data heterogeneity. This design offers practical advantages for privacy-preserving cross-institution learning in heterogeneous environments, with clear paths to extending to multi-party settings in future work.

Abstract

With the rapid advancement of the digital economy, data collaboration between organizations has become a well-established business model, driving the growth of various industries. However, privacy concerns make direct data sharing impractical. To address this, Two-Party Split Learning (a.k.a. Vertical Federated Learning (VFL)) has emerged as a promising solution for secure collaborative learning. Despite its advantages, this architecture still suffers from low computational resource utilization and training efficiency. Specifically, its synchronous dependency design increases training latency, while resource and data heterogeneity among participants further hinder efficient computation. To overcome these challenges, we propose PubSub-VFL, a novel VFL paradigm with a Publisher/Subscriber architecture optimized for two-party collaborative learning with high computational efficiency. PubSub-VFL leverages the decoupling capabilities of the Pub/Sub architecture and the data parallelism of the parameter server architecture to design a hierarchical asynchronous mechanism, reducing training latency and improving system efficiency. Additionally, to mitigate the training imbalance caused by resource and data heterogeneity, we formalize an optimization problem based on participants' system profiles, enabling the selection of optimal hyperparameters while preserving privacy. We conduct a theoretical analysis to demonstrate that PubSub-VFL achieves stable convergence and is compatible with security protocols such as differential privacy. Extensive case studies on five benchmark datasets further validate its effectiveness, showing that, compared to state-of-the-art baselines, PubSub-VFL not only accelerates training by without compromising accuracy, but also achieves a computational resource utilization rate of up to 91.07%.
Paper Structure (21 sections, 1 theorem, 41 equations, 8 figures, 10 tables, 2 algorithms)

This paper contains 21 sections, 1 theorem, 41 equations, 8 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. System and Algorithm Design
  5. System Design
  6. System Profiling
  7. System Planning Phase
  8. Experiment
  9. Experiment Setup
  10. Numerical Results
  11. Conclusion
  12. Scarecrow Solution
  13. Publisher/Subscriber Architecture
  14. Differential Privacy Protocol
  15. Convergence Proof
  16. ...and 6 more sections

Key Result

Theorem D.1

If Assumptions ass-1--ass-6 are held and if the update rule is applied with a constant learning rate $\eta$ chosen sufficiently small (so that higher-order terms are negligible), there exist constants such that the expected optimality gap satisfies where $\theta^\ast$ is the unique minimizer of $f(\theta)$.

Figures (8)

  • Figure 1: Overview of the split learning (SL)-based VFL framework.
  • Figure 2: System architecture and workflow of PubSub-VFL.
  • Figure 3: Comparison with existing baselines in computation and communication efficiency.
  • Figure 4: Comparison with existing baselines on computation efficiency in resource and data heterogeneous scenarios.
  • Figure 5: The impact of privacy budget on the performance, efficiency, and security of PubSub-VFL.
  • ...and 3 more figures

Theorems & Definitions (3)

  • Definition C.1
  • Theorem D.1
  • Proof D.1