Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedHypeVAE: Federated Learning with Hypernetwork Generated Conditional VAEs for Differentially Private Embedding Sharing

Sunny Gupta, Amit Sethi

TL;DR

FedHypeVAE addresses privacy-preserving embedding sharing under non-IID federated data by introducing a hypernetwork that generates client-specific CVAE decoders and class priors from private codes, trained with differential privacy. It couples this personalization with cross-site MMD alignment and a meta-code synthesis mechanism to maintain global coherence and domain coverage. The approach achieves strong DP guarantees via DP-SGD and demonstrates improved robustness and utility on medical imaging benchmarks, outperforming DP-CVAE and other baselines. This work establishes a scalable, domain-adaptive, privacy-preserving framework for collaborating across medical institutions without exposing raw data or shared model updates.

Abstract

Federated data sharing promises utility without centralizing raw data, yet existing embedding-level generators struggle under non-IID client heterogeneity and provide limited formal protection against gradient leakage. We propose FedHypeVAE, a differentially private, hypernetwork-driven framework for synthesizing embedding-level data across decentralized clients. Building on a conditional VAE backbone, we replace the single global decoder and fixed latent prior with client-aware decoders and class-conditional priors generated by a shared hypernetwork from private, trainable client codes. This bi-level design personalizes the generative layerrather than the downstream modelwhile decoupling local data from communicated parameters. The shared hypernetwork is optimized under differential privacy, ensuring that only noise-perturbed, clipped gradients are aggregated across clients. A local MMD alignment between real and synthetic embeddings and a Lipschitz regularizer on hypernetwork outputs further enhance stability and distributional coherence under non-IID conditions. After training, a neutral meta-code enables domain agnostic synthesis, while mixtures of meta-codes provide controllable multi-domain coverage. FedHypeVAE unifies personalization, privacy, and distribution alignment at the generator level, establishing a principled foundation for privacy-preserving data synthesis in federated settings. Code: github.com/sunnyinAI/FedHypeVAE

FedHypeVAE: Federated Learning with Hypernetwork Generated Conditional VAEs for Differentially Private Embedding Sharing

TL;DR

FedHypeVAE addresses privacy-preserving embedding sharing under non-IID federated data by introducing a hypernetwork that generates client-specific CVAE decoders and class priors from private codes, trained with differential privacy. It couples this personalization with cross-site MMD alignment and a meta-code synthesis mechanism to maintain global coherence and domain coverage. The approach achieves strong DP guarantees via DP-SGD and demonstrates improved robustness and utility on medical imaging benchmarks, outperforming DP-CVAE and other baselines. This work establishes a scalable, domain-adaptive, privacy-preserving framework for collaborating across medical institutions without exposing raw data or shared model updates.

Abstract

Federated data sharing promises utility without centralizing raw data, yet existing embedding-level generators struggle under non-IID client heterogeneity and provide limited formal protection against gradient leakage. We propose FedHypeVAE, a differentially private, hypernetwork-driven framework for synthesizing embedding-level data across decentralized clients. Building on a conditional VAE backbone, we replace the single global decoder and fixed latent prior with client-aware decoders and class-conditional priors generated by a shared hypernetwork from private, trainable client codes. This bi-level design personalizes the generative layerrather than the downstream modelwhile decoupling local data from communicated parameters. The shared hypernetwork is optimized under differential privacy, ensuring that only noise-perturbed, clipped gradients are aggregated across clients. A local MMD alignment between real and synthetic embeddings and a Lipschitz regularizer on hypernetwork outputs further enhance stability and distributional coherence under non-IID conditions. After training, a neutral meta-code enables domain agnostic synthesis, while mixtures of meta-codes provide controllable multi-domain coverage. FedHypeVAE unifies personalization, privacy, and distribution alignment at the generator level, establishing a principled foundation for privacy-preserving data synthesis in federated settings. Code: github.com/sunnyinAI/FedHypeVAE
Paper Structure (26 sections, 15 equations, 1 figure, 1 table, 1 algorithm)

This paper contains 26 sections, 15 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Gradient inversion and privacy in federated learning
  4. Privacy-preserving techniques in federated learning
  5. Federated and Differentially-Private Generative Models
  6. Hypernetworks for Federated Learning
  7. Problem Setup and Motivation
  8. Methodology
  9. Client-Level Conditional Generative Objective
  10. Hypernetwork-Based Parameter Generation
  11. Row-Scaled Efficient Generation.
  12. Hyper-Generated Class Priors.
  13. Stability Regularization and Cross-Site Alignment
  14. Cross-Site Distribution Alignment.
  15. Federated Hypernetwork Optimization under Differential Privacy
  16. ...and 11 more sections

Figures (1)

  • Figure 1: Overview of the proposed FedHypeVAE framework. (1) Each participating client $\mathcal{H}_i$ transforms its local image dataset $\mathcal{D}_i$ into an embedding-level dataset $\mathcal{S}_i$ using a frozen foundation encoder $\Phi$, substantially reducing communication and storage cost. (2) Locally, each client trains a conditional variational autoencoder (CVAE) parameterized by an encoder–decoder pair $(q_{\psi_i}, p_{\theta_i})$ and a class-conditional prior $p_{\omega_i}$, which model the embedding distribution without exposing raw data. (3) A server-side hypernetwork $H_{\Phi}=\{h_{\theta},h_{\omega}\}$ maps private client codes $v_i$ to client-specific decoder and prior parameters, and is optimized federatively via differentially-private stochastic gradient descent (DP-SGD). (4) After convergence, a neutral meta-code $v_{\circ}$ produces a global decoder–prior pair $(\theta_{\circ}, \omega_{\circ})$ that generates synthetic embeddings $\hat{\mathcal{S}}=\{(\hat{x},\hat{y})\}$, which can be combined with local data for downstream models $f_1,\ldots,f_N$.