De-VertiFL: A Solution for Decentralized Vertical Federated Learning
Alberto Huertas Celdrán, Chao Feng, Sabyasachi Banik, Gerome Bovet, Gregorio Martinez Perez, Burkhard Stiller
TL;DR
De-VertiFL addresses the challenge of decentralized Vertical Federated Learning by distributing neural network components across clients, exchanging hidden-layer outputs and gradients in a peer-to-peer manner without a central server. The approach combines forward-output sharing with FedAvg-style aggregation to build a global model from vertically partitioned data, implemented with MLPs in PyTorch and evaluated on MNIST, Fashion-MNIST, Titanic, and Bank Marketing. Across 2–10 participants and multi-class or binary tasks, De-VertiFL generally outperforms non-federated baselines and competitive centralized or decentralized methods, while highlighting how increased participation can degrade performance—a challenge mitigated by the proposed knowledge-exchange scheme. The results imply practical privacy-preserving collaboration for real-world VFL scenarios and motivate future work on robust aggregation and architectural innovations to sustain performance with many participants.
Abstract
Federated Learning (FL), introduced in 2016, was designed to enhance data privacy in collaborative model training environments. Among the FL paradigm, horizontal FL, where clients share the same set of features but different data samples, has been extensively studied in both centralized and decentralized settings. In contrast, Vertical Federated Learning (VFL), which is crucial in real-world decentralized scenarios where clients possess different, yet sensitive, data about the same entity, remains underexplored. Thus, this work introduces De-VertiFL, a novel solution for training models in a decentralized VFL setting. De-VertiFL contributes by introducing a new network architecture distribution, an innovative knowledge exchange scheme, and a distributed federated training process. Specifically, De-VertiFL enables the sharing of hidden layer outputs among federation clients, allowing participants to benefit from intermediate computations, thereby improving learning efficiency. De-VertiFL has been evaluated using a variety of well-known datasets, including both image and tabular data, across binary and multiclass classification tasks. The results demonstrate that De-VertiFL generally surpasses state-of-the-art methods in F1-score performance, while maintaining a decentralized and privacy-preserving framework.