Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios

Andrea Moleri, Christian Internò, Ali Raza, Markus Olhofer, David Klindt, Fabio Stella, Barbara Hammer

Abstract

Federated Learning (FL) enables distributed optimization without compromising data sovereignty. Yet, where local label distributions are mutually exclusive, standard weight aggregation fails due to conflicting optimization trajectories. Often, FL methods rely on pretrained foundation models, introducing unrealistic assumptions. We introduce FederatedFactory, a zero-dependency framework that inverts the unit of federation from discriminative parameters to generative priors. By exchanging generative modules in a single communication round, our architecture supports ex nihilo synthesis of universally class balanced datasets, eliminating gradient conflict and external prior bias entirely. Evaluations across diverse medical imagery benchmarks, including MedMNIST and ISIC2019, demonstrate that our approach recovers centralized upper-bound performance. Under pathological heterogeneity, it lifts baseline accuracy from a collapsed 11.36% to 90.57% on CIFAR-10 and restores ISIC2019 AUROC to 90.57%. Additionally, this framework facilitates exact modular unlearning through the deterministic deletion of specific generative modules.

FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios

Abstract

Federated Learning (FL) enables distributed optimization without compromising data sovereignty. Yet, where local label distributions are mutually exclusive, standard weight aggregation fails due to conflicting optimization trajectories. Often, FL methods rely on pretrained foundation models, introducing unrealistic assumptions. We introduce FederatedFactory, a zero-dependency framework that inverts the unit of federation from discriminative parameters to generative priors. By exchanging generative modules in a single communication round, our architecture supports ex nihilo synthesis of universally class balanced datasets, eliminating gradient conflict and external prior bias entirely. Evaluations across diverse medical imagery benchmarks, including MedMNIST and ISIC2019, demonstrate that our approach recovers centralized upper-bound performance. Under pathological heterogeneity, it lifts baseline accuracy from a collapsed 11.36% to 90.57% on CIFAR-10 and restores ISIC2019 AUROC to 90.57%. Additionally, this framework facilitates exact modular unlearning through the deterministic deletion of specific generative modules.
Paper Structure (26 sections, 2 theorems, 12 equations, 16 figures, 3 tables, 1 algorithm)

This paper contains 26 sections, 2 theorems, 12 equations, 16 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background and Mathematical Preliminaries
  4. Federated Learning under Single-Class Silo Regime.
  5. FederatedFactory Formal Problem Statement.
  6. Theoretical Guarantee of Zero-Dependency Convergence
  7. Methodology
  8. FederatedFactory
  9. Protocol A: Centralized Synthesis.
  10. Protocol B: Decentralized Synthesis.
  11. Multi-Class Generalization
  12. Modular Machine Unlearning and Exact Erasure
  13. Experiments and Results
  14. Experimental Setup.
  15. Results.
  16. ...and 11 more sections

Key Result

lemma 1

Under constraint C1 and Assumption 1, the KL divergence between the true global distribution and the zero-dependency synthetic distribution is strictly bounded by the weighted sum of the local diffusion errors: $\mathrm{KL}(p_{\mathrm{union}} \parallel \hat{p}_{\mathrm{syn}}) \le \sum_{k=1}^K \pi_k

Figures (16)

  • Figure 1: The Spectrum of Heterogeneity.(a) Ideal IID data (uniform overlap). (b) Dirichlet-distributed skew (imbalanced overlap). (c) Single-Class Silo (pathologically disjoint supports), representing the extreme theoretical limit.
  • Figure 2: Centralized FederatedFactory Protocol. Aggregated Factories produce a fully synthetic dataset $\hat{\mathcal{D}}$, enabling a global classifier training without raw data access.
  • Figure 3: Decentralized FederatedFactory Protocol. This architecture depicts the P2P topology where local data flows ($x \sim p_k(x)$) are augmented with synthetic samples from broadcasted Factories to train local experts, aggregated via PoE.
  • Figure 4: Modular Unlearning Modes in the Generative Matrix $\boldsymbol{\Gamma}$. By structuring the global model as a disjoint union of class-client generators $G_{\boldsymbol{\theta}_{c,k}}$, FederatedFactory enables exact erasure without retraining the entire ensemble.
  • Figure 5: Results in Pathological Heterogeneity. While standard baselines ( FedAvg $\circ$ , FedDyn $\square$ , FedProx $\triangle$ , Scaffold $\diamond$ ) collapse as we move from moderate skew ($\alpha=0.1$) to extreme silos ($\alpha \rightarrow 0$), FederatedFactory matches the Upper Bound (---) in both Centralized ($+$) , Decentralized ($\star$) configurations.
  • ...and 11 more figures

Theorems & Definitions (4)

  • lemma 1: Global Manifold Recovery
  • proof
  • theorem 1: Zero-Dependency Aggregation
  • proof