VFLAIR: A Research Library and Benchmark for Vertical Federated Learning

TL;DR

This work introduces VFLAIR, a lightweight and extensible framework and benchmark for Vertical Federated Learning (VFL) designed to track research progress and guide practical deployment. It covers 13 datasets, multiple model architectures, and two partition schemes, while benchmarking 11 attacks and 8 defenses under diverse communication protocols. The framework defines unified evaluation metrics, including the Defense Capability Scores (DCS, T-DCS, C-DCS), to quantify the trade-offs between task utility and privacy/security. Key findings show that splitVFL generally enhances robustness against attacks, that FedBCD and CELU-VFL can reduce communication rounds, and that defense choices should be tailored to attacker type and deployment scenario. The work also provides reproducibility resources and argues for broader adoption of standardized VFL benchmarks to accelerate safe, scalable real-world deployments.

Abstract

Vertical Federated Learning (VFL) has emerged as a collaborative training paradigm that allows participants with different features of the same group of users to accomplish cooperative training without exposing their raw data or model parameters. VFL has gained significant attention for its research potential and real-world applications in recent years, but still faces substantial challenges, such as in defending various kinds of data inference and backdoor attacks. Moreover, most of existing VFL projects are industry-facing and not easily used for keeping track of the current research progress. To address this need, we present an extensible and lightweight VFL framework VFLAIR (available at https://github.com/FLAIR-THU/VFLAIR), which supports VFL training with a variety of models, datasets and protocols, along with standardized modules for comprehensive evaluations of attacks and defense strategies. We also benchmark 11 attacks and 8 defenses performance under different communication and model partition settings and draw concrete insights and recommendations on the choice of defense strategies for different practical VFL deployment scenarios.

Figures (22)

• Figure 1: An overview of the Components of VFLAIR.
• Figure 2: A visual illustration example of DCS. The numbers on the contour lines are DCSs calculated with $\beta=0.5$.
• Figure 3: MPs and APs for different attacks under defenses [CIFAR10 dataset, aggVFL, FedSGD]
• Figure 4: Change of C-DCS ranking with the change of $\beta$. [MNIST dataset, aggVFL, FedSGD]
• Figure 5: DCS gap Distribution, y-axis represents density [MNIST dataset, splitVFL/aggVFL, FedSGD]