Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedHide: Federated Learning by Hiding in the Neighbors

Hyunsin Park, Sungrack Yun

TL;DR

A prototype-based federated learning method designed for embedding networks in classification or verification tasks and demonstrates its effectiveness through empirical results on three benchmark datasets: CIFAR-100, VoxCeleb1, and VGGFace2.

Abstract

We propose a prototype-based federated learning method designed for embedding networks in classification or verification tasks. Our focus is on scenarios where each client has data from a single class. The main challenge is to develop an embedding network that can distinguish between different classes while adhering to privacy constraints. Sharing true class prototypes with the server or other clients could potentially compromise sensitive information. To tackle this issue, we propose a proxy class prototype that will be shared among clients instead of the true class prototype. Our approach generates proxy class prototypes by linearly combining them with their nearest neighbors. This technique conceals the true class prototype while enabling clients to learn discriminative embedding networks. We compare our method to alternative techniques, such as adding random Gaussian noise and using random selection with cosine similarity constraints. Furthermore, we evaluate the robustness of our approach against gradient inversion attacks and introduce a measure for prototype leakage. This measure quantifies the extent of private information revealed when sharing the proposed proxy class prototype. Moreover, we provide a theoretical analysis of the convergence properties of our approach. Our proposed method for federated learning from scratch demonstrates its effectiveness through empirical results on three benchmark datasets: CIFAR-100, VoxCeleb1, and VGGFace2.

FedHide: Federated Learning by Hiding in the Neighbors

TL;DR

A prototype-based federated learning method designed for embedding networks in classification or verification tasks and demonstrates its effectiveness through empirical results on three benchmark datasets: CIFAR-100, VoxCeleb1, and VGGFace2.

Abstract

We propose a prototype-based federated learning method designed for embedding networks in classification or verification tasks. Our focus is on scenarios where each client has data from a single class. The main challenge is to develop an embedding network that can distinguish between different classes while adhering to privacy constraints. Sharing true class prototypes with the server or other clients could potentially compromise sensitive information. To tackle this issue, we propose a proxy class prototype that will be shared among clients instead of the true class prototype. Our approach generates proxy class prototypes by linearly combining them with their nearest neighbors. This technique conceals the true class prototype while enabling clients to learn discriminative embedding networks. We compare our method to alternative techniques, such as adding random Gaussian noise and using random selection with cosine similarity constraints. Furthermore, we evaluate the robustness of our approach against gradient inversion attacks and introduce a measure for prototype leakage. This measure quantifies the extent of private information revealed when sharing the proposed proxy class prototype. Moreover, we provide a theoretical analysis of the convergence properties of our approach. Our proposed method for federated learning from scratch demonstrates its effectiveness through empirical results on three benchmark datasets: CIFAR-100, VoxCeleb1, and VGGFace2.
Paper Structure (17 sections, 42 equations, 7 figures, 4 tables, 1 algorithm)

This paper contains 17 sections, 42 equations, 7 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Federated Learning Based on a Contrastive Learning Loss with Proxy Prototypes
  5. Proxy Class Prototype Generation by Hiding in the Neighbors
  6. Convergence Analysis
  7. Prototype Leakage Measure
  8. Experiments
  9. Experimental Setup
  10. Results
  11. Discussion
  12. Conclusion
  13. Convergence Analysis for FedHide
  14. Assumptions
  15. Key Lemmas
  16. ...and 2 more sections

Figures (7)

  • Figure 1: A diagram of the FedHide algorithm. Each client updates its local embedding network and prototype using a contrastive loss and shared proxy prototypes. The server collects the local updates and proxy prototypes, then broadcasts the aggregated model parameters and proxy prototypes to all clients.
  • Figure 1: t-SNE visualizations of FedGN methods for the CIFAR-100 dataset. (red circle: true class prototype, green square: proxy class prototype, dashed line: pairs of true and proxy prototypes)
  • Figure 2: Three proxy class prototype generation methods. The negative loss is applied using the proxy class prototype $\bar{w}_c$ and other client embedding $w_{c'}$
  • Figure 2: t-SNE visualizations of FedCS methods for the CIFAR-100 dataset. (red circle: true class prototype, green square: proxy class prototype, dashed line: pairs of true and proxy class prototypes)
  • Figure 3: FedHide results on CIFAR-100. Subfigure (a), (b), and (c) show the accuracy curves with different privacy control parameters. Subfigure (d) shows that the FedHide is the best in terms of high accuracy and low prototype leakage.
  • ...and 2 more figures

Theorems & Definitions (3)

  • proof
  • proof
  • proof