Decoupled Vertical Federated Learning for Practical Training on Vertically Partitioned Data

TL;DR

DVFL introduces a decoupled approach to Vertical Federated Learning that eliminates forward/backward locking by enabling asynchronous, locally supervised training at guests and hosts, plus a label-owner transfer learner. The three-tier DVFL hierarchy uses per-guest registers and input replay to support fault tolerance, redundancy, and learning from data beyond the intersection, while preserving gradient privacy. Empirical results show DVFL achieves graceful degradation under faults, with redundancy further boosting performance, and it can outperform traditional VFL baselines on several vertically partitioned datasets. The method offers engineering flexibility in communication and computation, enabling scalable, privacy-preserving learning for vertically partitioned data in realistic, imperfect networks.

Abstract

Vertical Federated Learning (VFL) is an emergent distributed machine learning paradigm for collaborative learning between clients who have disjoint features of common entities. However, standard VFL lacks fault tolerance, with each participant and connection being a single point of failure. Prior attempts to induce fault tolerance in VFL focus on the scenario of "straggling clients", usually entailing that all messages eventually arrive or that there is an upper bound on the number of late messages. To handle the more general problem of arbitrary crashes, we propose Decoupled VFL (DVFL). To handle training with faults, DVFL decouples training between communication rounds using local unsupervised objectives. By further decoupling label supervision from aggregation, DVFL also enables redundant aggregators. As secondary benefits, DVFL can enhance data efficiency and provides immunity against gradient-based attacks. In this work, we implement DVFL for split neural networks with a self-supervised autoencoder loss. When there are faults, DVFL outperforms the best VFL-based alternative (97.58% vs 96.95% on an MNIST task). Even under perfect conditions, performance is comparable.

Figures (5)

• Figure 1: Diagram illustrating the distributed training of a neural network under SplitNN, VFL with SplitNN, and DVFL. The two input data for VFL and DVFL are partial features of the same entity.
• Figure 2: A DVFL system with three guests (a) and two hosts (b). Guests train their local models on unsupervised objectives and hosts also train their aggregating models on unsupervised objectives. After that, the label owner (c) trains a transfer learning model (such as a linear classifier head) on the encodings from the hosts.
• Figure 3: Model performance degrades with an increase in communication period, i.e. more communication is correlated with better performance.
• Figure 4: Model performance degrades with an increase in communication period, i.e. more communication is correlated with better performance.
• Figure :