Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning on Riemannian Manifolds with Differential Privacy

Zhenwei Huang, Wen Huang, Pratik Jawanpuria, Bamdev Mishra

TL;DR

The paper tackles privacy-preserving federated learning when model parameters live on non-Euclidean spaces, namely Riemannian manifolds. It introduces PriRFed, a generic framework that enforces differential privacy through privately trained local updates and aggregates on the manifold, achieving global DP guarantees. The authors provide convergence analyses for two privately local training schemes, DP-RSGD and DP-RSVRG, covering nonconvex and convex settings, and validate the approach on principal eigenvector computation on the sphere, Fréchet mean computation on SPD manifolds, and hyperbolic structured prediction. The results demonstrate a clear privacy-utility tradeoff and show that DP-RSVRG generally offers faster convergence and better robustness to noise than DP-RSGD in manifold contexts, highlighting PriRFed’s practical relevance for privacy-sensitive, geometry-aware distributed learning.

Abstract

In recent years, federated learning (FL) has emerged as a prominent paradigm in distributed machine learning. Despite the partial safeguarding of agents' information within FL systems, a malicious adversary can potentially infer sensitive information through various means. In this paper, we propose a generic private FL framework defined on Riemannian manifolds (PriRFed) based on the differential privacy (DP) technique. We analyze the privacy guarantee while establishing the convergence properties. To the best of our knowledge, this is the first federated learning framework on Riemannian manifold with a privacy guarantee and convergence results. Numerical simulations are performed on synthetic and real-world datasets to showcase the efficacy of the proposed PriRFed approach.

Federated Learning on Riemannian Manifolds with Differential Privacy

TL;DR

The paper tackles privacy-preserving federated learning when model parameters live on non-Euclidean spaces, namely Riemannian manifolds. It introduces PriRFed, a generic framework that enforces differential privacy through privately trained local updates and aggregates on the manifold, achieving global DP guarantees. The authors provide convergence analyses for two privately local training schemes, DP-RSGD and DP-RSVRG, covering nonconvex and convex settings, and validate the approach on principal eigenvector computation on the sphere, Fréchet mean computation on SPD manifolds, and hyperbolic structured prediction. The results demonstrate a clear privacy-utility tradeoff and show that DP-RSVRG generally offers faster convergence and better robustness to noise than DP-RSGD in manifold contexts, highlighting PriRFed’s practical relevance for privacy-sensitive, geometry-aware distributed learning.

Abstract

In recent years, federated learning (FL) has emerged as a prominent paradigm in distributed machine learning. Despite the partial safeguarding of agents' information within FL systems, a malicious adversary can potentially infer sensitive information through various means. In this paper, we propose a generic private FL framework defined on Riemannian manifolds (PriRFed) based on the differential privacy (DP) technique. We analyze the privacy guarantee while establishing the convergence properties. To the best of our knowledge, this is the first federated learning framework on Riemannian manifold with a privacy guarantee and convergence results. Numerical simulations are performed on synthetic and real-world datasets to showcase the efficacy of the proposed PriRFed approach.
Paper Structure (34 sections, 17 theorems, 60 equations, 6 figures, 1 table, 3 algorithms)

This paper contains 34 sections, 17 theorems, 60 equations, 6 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. In Euclidean space.
  4. On manifolds.
  5. Our work and main contributions
  6. Outline
  7. Preliminaries
  8. Riemannian geometry.
  9. Function classes.
  10. Federated learning.
  11. Differential privacy.
  12. Private federated learning on Riemannian manifolds
  13. Privacy guarantee analysis
  14. Convergence analysis
  15. DP-RSGD for privately local training
  16. ...and 19 more sections

Key Result

Theorem 2.1

Suppose that $\mathcal{A}:\mathcal{Z}^n \rightarrow \mathcal{M}$ is a randomized algorithm that is $(\epsilon,\delta)$-DP. Let $P:\mathcal{M}\rightarrow\mathcal{M}'$ be an arbitrary mapping. Then $P\circ \mathcal{A}:\mathcal{Z}^n \rightarrow\mathcal{M}'$ is $(\epsilon,\delta)$-DP.

Figures (6)

  • Figure 1: Averaged results over $10$ tests for Problem \ref{['NumExp:1']} using PriRFed-DP-RSGD and PriRFed-DP-RSVRG with different private levels. Here $d+1=25$, $v = 10^{-3}$. The $y$-axis of the figures in the first row is the cost value. And the one in the second row denotes $\|\mathrm{grad}f(x^{(t)})\|_{x^{(t)}}$. The $x$-axis of all figures means the iterations. For the first column, $s_t=N$, $K=1$, , $b_i=N_i$ and $\alpha_t=1/(2L_g)$. For the second column, $s_t=1$, $K=5$, $b_i=N_i/2$, and $\alpha_t=1.0$. In the first two columns, DP-RSGD is used as the privately local training procedure, and DP-RSVRG in the third. For the third column, $s_t=1$, $K=2$, $m=\lfloor 10N/3 \rfloor$, and $\alpha_t=1/(10N^{2/3}L_g)$.
  • Figure 2: Averaged results over $10$ tests for Problem \ref{['NumExp:1']} with MNIST dataset. The legends "DP-RSGD" and "DP-SRSGD" refer respectively to PriRFed-DP-RSGD and PriRFed-DP-RSVRG, and "True Value" is provided by Riemannian steepest descent method.
  • Figure 3: Averaged results over $10$ tests for Problem \ref{['NumExp:2']} with synthetic dataset. The legends "DP-RSGD" and "DP-SRSGD" refer respectively to PriRFed-DP-RSGD and PriRFed-DP-RSVRG, and "True Value" is provided by Riemannian steepest gradient method. Excess risk is defined by $f(\tilde{x})-f(x^*)$, where $f(x^*)$ is given through steepest descent method.
  • Figure 4: Averaged results over $10$ tests for Problem \ref{['NumExp:2']} with PATHMNIST dataset. The legends "DP-RSGD" and "DP-SRSGD" refer respectively to PriRFed-DP-RSGD and PriRFed-DP-RSVRG, and "True Value" is provided by Riemannian steepest descent method. Excess risk is defined by $f(\tilde{x})-f(x^*)$, where $f(x^*)$ is given through steepest descent method.
  • Figure 5: Averaged results over $10$ tests for Problem \ref{['NumExp:3']} with WordNet dataset. The legends "RSGD", "DP-RSGD", "RSVRG" and "DP-SRSGD" refer respectively to PriRFed-RSGD, PriRFed-DP-RSGD, PriRFed-RSVRG and PriRFed-DP-RSVRG, respectively. (a) The values of cost against iterations. (b) The Riemannian distance against iterations.
  • ...and 1 more figures

Theorems & Definitions (34)

  • Definition 2.1: Geodesic Lipschitz continuity
  • Definition 2.2: Geodesic smoothness
  • Definition 2.3: Geodesic convexity ZS16
  • Definition 2.4: Differential privacy DR14
  • Theorem 2.1: Post-processing DR14
  • Theorem 2.2: Sequential composition theorem DR14
  • Theorem 2.3: Advanced composition theorem DRV10KOV17WRRW23
  • Definition 2.5: Tangent space Gaussian distribution HMJG22UHJM23
  • Definition 4.1: Subsample
  • Lemma 4.1: Subsampling lemma
  • ...and 24 more