Vertical Federated Learning: Challenges, Methodologies and Experiments

TL;DR

Vertical Federated Learning (VFL) enables collaborative modeling with vertically partitioned attributes but introduces unique privacy, computation, and heterogeneity challenges not present in horizontal FL. The paper proposes a general VFL framework and analyzes four core challenges, then develops privacy-preserving, communication-efficient, asynchronous, and splitting-design methodologies, validated through DP-assisted VFL, compression-based communication, and resource-aware splitting experiments. Key contributions include formalizing the seven-step PSI-based VFL workflow, elucidating privacy-utility trade-offs with differential privacy, and providing empirical guidance on how compression and splitting depth affect cost and accuracy. The findings offer practical guidance for deploying VFL in real-world settings with sensitive attribute data and constrained communication resources, highlighting actionable design choices for privacy, efficiency, and robustness.

Abstract

Recently, federated learning (FL) has emerged as a promising distributed machine learning (ML) technology, owing to the advancing computational and sensing capacities of end-user devices, however with the increasing concerns on users' privacy. As a special architecture in FL, vertical FL (VFL) is capable of constructing a hyper ML model by embracing sub-models from different clients. These sub-models are trained locally by vertically partitioned data with distinct attributes. Therefore, the design of VFL is fundamentally different from that of conventional FL, raising new and unique research issues. In this paper, we aim to discuss key challenges in VFL with effective solutions, and conduct experiments on real-life datasets to shed light on these issues. Specifically, we first propose a general framework on VFL, and highlight the key differences between VFL and conventional FL. Then, we discuss research challenges rooted in VFL systems under four aspects, i.e., security and privacy risks, expensive computation and communication costs, possible structural damage caused by model splitting, and system heterogeneity. Afterwards, we develop solutions to addressing the aforementioned challenges, and conduct extensive experiments to showcase the effectiveness of our proposed solutions.

Figures (6)

• Figure 1: A diagram for Vertical FL. The classic workflow includes following seven steps: 1) private set intersection; 2) bottom model forward propagation (BM-FP); 3) forward transmission; 4) top model forward propagation (TM-FP); 5) top model backward propagation (TM-BP); 6) backward transmission; 7) bottom model backward propagation (BM-BP). The host is the label owner and the guest is the attribute owner.
• Figure 2: A diagram for VFL with the compression function. The compression function can be viewed as an additive layer and it is usually not differentiable, and thus an approximate function can be used to compensate compression errors.
• Figure 3: AUC with different privacy levels in the VFL framework with three participants (Avazu dataset).
• Figure 4: The test AUC with different compression levels in the VFL framework (Adult dataset).
• Figure 5: The test AUC with different approximation functions under the same compression level in the VFL framework (Adult dataset).