Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fully Decentralized Certified Unlearning

Hithem Lamri, Michail Maniatakos

TL;DR

The paper tackles certified unlearning in fully decentralized networks with fixed topologies by introducing RR-DU, a random-walk based unlearning algorithm. RR-DU localizes noise to the unlearning client and leverages randomized routing, local averaging, and a trust-region constraint to achieve strong privacy-utility trade-offs, with network-DP view-based certificates and convergence guarantees. The authors establish two regimes for deletion capacity, show that RR-DU avoids the forget-set size dependence that plagues decentralized DP baselines, and provide both theoretical results (last-iterate guarantees, stationarity, and deletion-capacity bounds) and empirical validation on MNIST and CIFAR-10 demonstrating near-scratch forgetting with superior retained accuracy. These contributions enable practical, scalable certified unlearning in decentralized settings, with explicit guidance on routing, averaging, and noise calibration for real-world deployment.

Abstract

Machine unlearning (MU) seeks to remove the influence of specified data from a trained model in response to privacy requests or data poisoning. While certified unlearning has been analyzed in centralized and server-orchestrated federated settings (via guarantees analogous to differential privacy, DP), the decentralized setting -- where peers communicate without a coordinator remains underexplored. We study certified unlearning in decentralized networks with fixed topologies and propose RR-DU, a random-walk procedure that performs one projected gradient ascent step on the forget set at the unlearning client and a geometrically distributed number of projected descent steps on the retained data elsewhere, combined with subsampled Gaussian noise and projection onto a trust region around the original model. We provide (i) convergence guarantees in the convex case and stationarity guarantees in the nonconvex case, (ii) $(\varepsilon,δ)$ network-unlearning certificates on client views via subsampled Gaussian Rényi DP (RDP) with segment-level subsampling, and (iii) deletion-capacity bounds that scale with the forget-to-local data ratio and quantify the effect of decentralization (network mixing and randomized subsampling) on the privacy-utility trade-off. Empirically, on image benchmarks (MNIST, CIFAR-10), RR-DU matches a given $(\varepsilon,δ)$ while achieving higher test accuracy than decentralized DP baselines and reducing forget accuracy to random guessing ($\approx 10\%$).

Fully Decentralized Certified Unlearning

TL;DR

The paper tackles certified unlearning in fully decentralized networks with fixed topologies by introducing RR-DU, a random-walk based unlearning algorithm. RR-DU localizes noise to the unlearning client and leverages randomized routing, local averaging, and a trust-region constraint to achieve strong privacy-utility trade-offs, with network-DP view-based certificates and convergence guarantees. The authors establish two regimes for deletion capacity, show that RR-DU avoids the forget-set size dependence that plagues decentralized DP baselines, and provide both theoretical results (last-iterate guarantees, stationarity, and deletion-capacity bounds) and empirical validation on MNIST and CIFAR-10 demonstrating near-scratch forgetting with superior retained accuracy. These contributions enable practical, scalable certified unlearning in decentralized settings, with explicit guidance on routing, averaging, and noise calibration for real-world deployment.

Abstract

Machine unlearning (MU) seeks to remove the influence of specified data from a trained model in response to privacy requests or data poisoning. While certified unlearning has been analyzed in centralized and server-orchestrated federated settings (via guarantees analogous to differential privacy, DP), the decentralized setting -- where peers communicate without a coordinator remains underexplored. We study certified unlearning in decentralized networks with fixed topologies and propose RR-DU, a random-walk procedure that performs one projected gradient ascent step on the forget set at the unlearning client and a geometrically distributed number of projected descent steps on the retained data elsewhere, combined with subsampled Gaussian noise and projection onto a trust region around the original model. We provide (i) convergence guarantees in the convex case and stationarity guarantees in the nonconvex case, (ii) network-unlearning certificates on client views via subsampled Gaussian Rényi DP (RDP) with segment-level subsampling, and (iii) deletion-capacity bounds that scale with the forget-to-local data ratio and quantify the effect of decentralization (network mixing and randomized subsampling) on the privacy-utility trade-off. Empirically, on image benchmarks (MNIST, CIFAR-10), RR-DU matches a given while achieving higher test accuracy than decentralized DP baselines and reducing forget accuracy to random guessing ().

Paper Structure

This paper contains 77 sections, 13 theorems, 79 equations, 1 figure, 17 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Private Decentralized Learning
  4. Certified Unlearning
  5. Problem Statement
  6. Decentralized Formulation.
  7. Unlearning via Differential Privacy.
  8. Proof sketch.
  9. Decentralization Effect on Deletion Capacity.
  10. Algorithm Design
  11. Motivation and Problem Setup
  12. Mathematical Foundation: Gradient Alignment
  13. Decentralized Realization via Random Walk
  14. Exact vs. Lightweight Alignment at u
  15. Synthesis of Advantages
  16. ...and 62 more sections

Key Result

Theorem 3.1

Let $\Theta\subset\mathbb{R}^{d}$ be convex with diameter $R:=\sup_{\theta,\theta'\in\Theta}\|\theta-\theta'\|_{2}$, and assume the loss $\ell$ is $L$-smooth and convex. Consider the network-private SGD (Appendix B.2), run for $T$ hops with $N$ clients, and take $\mathcal{U}(D_f,\mathcal{A}(D),T(D))

Figures (1)

  • Figure 1: Backdoor unlearning results on MNIST and CIFAR-10. RR-DU vs. finetuning, DPSGD, and DDP. Vertical dashed lines mark unlearning start; horizontal dashed lines denote scratch baselines.

Theorems & Definitions (30)

  • Definition 1: $(\varepsilon,\delta)$-Certified Unlearning
  • Definition 2: Network Differential Privacy DBLP:conf/aistats/CyffersB22
  • Definition 3: $(\varepsilon,\delta)$-Decentralized Certified Unlearning
  • Definition 4: Deletion capacity
  • Theorem 3.1: Deletion capacity of decentralized DP
  • Theorem 5.1: $(\varepsilon,\delta)$-DCU via view-based amplification
  • Corollary 1: Noise calibration (scaling)
  • Definition 5: $(\varepsilon,\delta)$-Certified Unlearning (global)
  • Definition 6: Network Differential Privacy (Network-DP)
  • Definition 7: $(\varepsilon,\delta)$-Decentralized Certified Unlearning
  • ...and 20 more