FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Training

TL;DR

This article proposes Federated Cross-view Training (FedCVT), a semi-supervised learning approach that improves the performance of the VFL model with limited aligned samples and significantly outperforms vanilla VFL that only utilizes aligned samples.

Abstract

Federated learning allows multiple parties to build machine learning models collaboratively without exposing data. In particular, vertical federated learning (VFL) enables participating parties to build a joint machine learning model based on distributed features of aligned samples. However, VFL requires all parties to share a sufficient amount of aligned samples. In reality, the set of aligned samples may be small, leaving the majority of the non-aligned data unused. In this article, we propose Federated Cross-view Training (FedCVT), a semi-supervised learning approach that improves the performance of the VFL model with limited aligned samples. More specifically, FedCVT estimates representations for missing features, predicts pseudo-labels for unlabeled samples to expand the training set, and trains three classifiers jointly based on different views of the expanded training set to improve the VFL model's performance. FedCVT does not require parties to share their original data and model parameters, thus preserving data privacy. We conduct experiments on NUS-WIDE, Vehicle, and CIFAR10 datasets. The experimental results demonstrate that FedCVT significantly outperforms vanilla VFL that only utilizes aligned samples. Finally, we perform ablation studies to investigate the contribution of each component of FedCVT to the performance of FedCVT. Code is available at https://github.com/yankang18/FedCVT

Figures (8)

• Figure 1: View of the virtual dataset in vertical federated learning. Each party owns a vertical partition (or a partial view) of this dataset.
• Figure 2: Overview of FedCVT approach
• Figure 3: Learn representations from raw input features. Each party $p \in \{A, B\}$ has two neural network models $\it{h}_u^p$ and $\it{h}_c^p$ to learn unique and shared representations, respectively.
• Figure 4: Estimate missing representations $\tilde{R}^{A}$ corresponds to the non-aligned representations $R^{B}_{nl}$ of party B. (a) estimates missing representations $\tilde{R}^{A}_{c}$ corresponding to shared representations $R^{B}_{nl,c}$ while (b) estimates missing representations $\tilde{R}^{A}_{u}$ corresponding to unique representations $R^{B}_{nl,u}$. Then, $\tilde{R}^{A}$ can be obtained by concatenating $\tilde{R}^{A}_{c}$ and $\tilde{R}^{A}_{u}$.
• Figure 5: View of the training set $\chi$, each sample of which consists of the representation of a raw input feature and its corresponding label. (a) The view of $\chi$ with missing representations estimated. (b) The view of $\chi$ with cherry-picked pseudo-labeled samples.