Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Latent Variable Model based Vertical Federated Learning with Flexible Alignment and Labeling Scenarios

Kihun Hong, Sejun Park, Ganguk Hwang

TL;DR

This work reinterpret alignment gaps in VFL as missing data problems and proposes a unified framework that accommodates both training and inference under arbitrary alignment and labeling scenarios, while supporting diverse missingness mechanisms.

Abstract

Federated learning (FL) has attracted significant attention for enabling collaborative learning without exposing private data. Among the primary variants of FL, vertical federated learning (VFL) addresses feature-partitioned data held by multiple institutions, each holding complementary information for the same set of users. However, existing VFL methods often impose restrictive assumptions such as a small number of participating parties, fully aligned data, or only using labeled data. In this work, we reinterpret alignment gaps in VFL as missing data problems and propose a unified framework that accommodates both training and inference under arbitrary alignment and labeling scenarios, while supporting diverse missingness mechanisms. In the experiments on 168 configurations spanning four benchmark datasets, six training-time missingness patterns, and seven testing-time missingness patterns, our method outperforms all baselines in 160 cases with an average gap of 9.6 percentage points over the next-best competitors. To the best of our knowledge, this is the first VFL framework to jointly handle arbitrary data alignment, unlabeled data, and multi-party collaboration all at once.

Deep Latent Variable Model based Vertical Federated Learning with Flexible Alignment and Labeling Scenarios

TL;DR

This work reinterpret alignment gaps in VFL as missing data problems and proposes a unified framework that accommodates both training and inference under arbitrary alignment and labeling scenarios, while supporting diverse missingness mechanisms.

Abstract

Federated learning (FL) has attracted significant attention for enabling collaborative learning without exposing private data. Among the primary variants of FL, vertical federated learning (VFL) addresses feature-partitioned data held by multiple institutions, each holding complementary information for the same set of users. However, existing VFL methods often impose restrictive assumptions such as a small number of participating parties, fully aligned data, or only using labeled data. In this work, we reinterpret alignment gaps in VFL as missing data problems and propose a unified framework that accommodates both training and inference under arbitrary alignment and labeling scenarios, while supporting diverse missingness mechanisms. In the experiments on 168 configurations spanning four benchmark datasets, six training-time missingness patterns, and seven testing-time missingness patterns, our method outperforms all baselines in 160 cases with an average gap of 9.6 percentage points over the next-best competitors. To the best of our knowledge, this is the first VFL framework to jointly handle arbitrary data alignment, unlabeled data, and multi-party collaboration all at once.
Paper Structure (40 sections, 1 theorem, 18 equations, 8 figures, 18 tables, 2 algorithms)

This paper contains 40 sections, 1 theorem, 18 equations, 8 figures, 18 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Our Method: FALSE-VFL
  5. Problem Setting and Notations
  6. Problem and Approach
  7. Algorithm
  8. Setup
  9. Pretrainig with Marginal Likelihood Maximization
  10. Training with Conditional Likelihood Maximization
  11. Prediction
  12. Maximizing Conditional Likelihood through Two-Stage Optimization
  13. Two-Stage Optimization Strategy
  14. Interpretation as Feature Learning
  15. Incorporating Unlabeled Data
  16. ...and 25 more sections

Key Result

Theorem C.1

Let where Then, $\mathcal{L}_\kappa$$\left(\mathcal{L}_\kappa^{'}, \text{resp.}\right)$ increases as $\kappa$ increases, and bounded above by $\log p(\bm{x}^{obs})$$\left(\log p(y,\bm{x}^{obs}), \text{resp.}\right)$. In addition, if $\frac{p(\bm{x}^{obs},\bm{h}, \bm{z})}{q(\bm{h}, \bm{z}|\bm{x}^{obs})}$$\left

Figures (8)

  • Figure 1: Left: Graphical model for FALSE-VFL-I. The left dotted box groups feature-side modules, while the right dotted box groups label-side modules. Right: Computational structure for FALSE-VFL-I.
  • Figure 2: Mean accuracy (%) of six VFL methods on each dataset trained under MNAR 7 conditions and tested across seven missing data patterns. Results are averaged over five independent runs; solid lines denote the mean accuracy and shaded regions indicate $\pm 1$ standard deviation.
  • Figure 3: Left: Graphical model for FALSE-VFL-II. The left dotted box groups feature-side modules, while the right dotted box groups label-side modules. Right: Computational structure for FALSE-VFL-II.
  • Figure 4: Mean accuracy (%) of six VFL methods on each dataset trained under MCAR 2 conditions and tested across seven missing data patterns. Results are averaged over five independent runs; solid lines denote the mean accuracy and shaded regions indicate $\pm 1$ standard deviation.
  • Figure 5: Mean accuracy (%) of six VFL methods on each dataset trained under MCAR 5 conditions and tested across seven missing data patterns. Results are averaged over five independent runs; solid lines denote the mean accuracy and shaded regions indicate $\pm 1$ standard deviation.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Theorem C.1
  • proof