Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CA-HFP: Curvature-Aware Heterogeneous Federated Pruning with Model Reconstruction

Gang Hu, Yinglei Teng, Pengfei Wu, Shijun Ma

Abstract

Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a practical framework that enables each client perform structured, device-specific pruning guided by a curvature-informed significance score, and subsequently maps its compact submodel back into a common global parameter space via a lightweight reconstruction. We derive a convergence bound for federated optimization with multiple local SGD steps that explicitly accounts for local computation, data heterogeneity, and pruning-induced perturbations; from which a principled loss-based pruning criterion is derived. Extensive experiments on FMNIST, CIFAR-10, and CIFAR-100 using VGG and ResNet architectures under varying degrees of data heterogeneity demonstrate that CA-HFP preserves model accuracy while significantly reducing per-client computation and communication costs, outperforming standard federated training and existing pruning-based baselines.

CA-HFP: Curvature-Aware Heterogeneous Federated Pruning with Model Reconstruction

Abstract

Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a practical framework that enables each client perform structured, device-specific pruning guided by a curvature-informed significance score, and subsequently maps its compact submodel back into a common global parameter space via a lightweight reconstruction. We derive a convergence bound for federated optimization with multiple local SGD steps that explicitly accounts for local computation, data heterogeneity, and pruning-induced perturbations; from which a principled loss-based pruning criterion is derived. Extensive experiments on FMNIST, CIFAR-10, and CIFAR-100 using VGG and ResNet architectures under varying degrees of data heterogeneity demonstrate that CA-HFP preserves model accuracy while significantly reducing per-client computation and communication costs, outperforming standard federated training and existing pruning-based baselines.
Paper Structure (19 sections, 2 theorems, 15 equations, 6 figures, 2 tables)

This paper contains 19 sections, 2 theorems, 15 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. System Model
  3. Heterogeneous Federated Pruning Framework
  4. Convergence Analysis
  5. Curvature-Aware Personal Pruning Design
  6. Loss-Perturbation Analysis Induced by Model Pruning
  7. Heterogeneous Model Pruning and Reconstruction
  8. Mask Generation Based on Curvature-Aware Loss Perturbation
  9. Model Reconstruction for Heterogeneous Sub-Models
  10. Experimental Evaluation
  11. Experiment Setup
  12. Comparison with the SOTA methods
  13. Performance Comparison of Proposed CA-HFP
  14. Convergence of CA-HFP
  15. Impact of Pruning Degree on CA-HFP
  16. ...and 4 more sections

Key Result

Lemma 1

Suppose that each client performs $E$ local SGD steps as in Eq. eq:local_sgd, and the server aggregates according to Eq. eq:agg. Let the local step size $0 < \eta \leqslant \frac{1}{{4LE}}$, and define $e_t$ implicitly by Eq. eq:global_update_noise. Under Assumptions as:smooth--as:heterogeneity, th

Figures (6)

  • Figure 1: Curvature-Aware Heterogeneous Federated Pruning (CA-HFP) Framework.
  • Figure 2: Model Pruning and Reconstruction.
  • Figure 3: The convergence with different pruning methods.
  • Figure 4: The accuracy under different pruning ratio.
  • Figure 5: The cost under different pruning ratio. The communication cost includes the uploading and the downloading and the calculation of FLOPS is according to PruneFL.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Lemma 1: Result of one round
  • Theorem 1: FL Convergence with personal pruning