Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lightweight Federated Learning in Mobile Edge Computing with Statistical and Device Heterogeneity Awareness

Jinghong Tan, Zhichen Zhang, Kun Guo, Tsung-Hui Chang, Tony Q. S. Quek

TL;DR

This work tackles federated learning in resource-constrained MEC under statistical and device heterogeneity by introducing a lightweight personalized FL framework built on parameter decoupling. It anatomizes the model into base and personalization subspaces, applying gradient sparsification to the shared base and pruning to the private personalization to decouple communication and computation costs from personalization quality. A rigorous convergence analysis reveals how sparsification and pruning jointly affect iteration complexity, leading to a joint optimization across per-client sparsity, pruning, and wireless bandwidth to minimize end-to-end training time. Empirical results on image and NLP tasks demonstrate faster convergence and substantial cost reductions with minimal accuracy loss, validating the practical value of coordinated resource-aware personalization in heterogeneous MEC environments.

Abstract

Federated learning enables collaborative machine learning while preserving data privacy, but high communication and computation costs, exacerbated by statistical and device heterogeneity, limit its practicality in mobile edge computing. Existing compression methods like sparsification and pruning reduce per-round costs but may increase training rounds and thus the total training cost, especially under heterogeneous environments. We propose a lightweight personalized FL framework built on parameter decoupling, which separates the model into shared and private subspaces, enabling us to uniquely apply gradient sparsification to the shared component and model pruning to the private one. This structural separation confines communication compression to global knowledge exchange and computation reduction to local personalization, protecting personalization quality while adapting to heterogeneous client resources. We theoretically analyze convergence under the combined effects of sparsification and pruning, revealing a sparsity-pruning trade-off that links to the iteration complexity. Guided by this analysis, we formulate a joint optimization that selects per-client sparsity and pruning rates and wireless bandwidth to reduce end-to-end training time. Simulation results demonstrate faster convergence and substantial reductions in overall communication and computation costs with negligible accuracy loss, validating the benefits of coordinated and resource-aware personalization in resource-constrained heterogeneous environments.

Lightweight Federated Learning in Mobile Edge Computing with Statistical and Device Heterogeneity Awareness

TL;DR

This work tackles federated learning in resource-constrained MEC under statistical and device heterogeneity by introducing a lightweight personalized FL framework built on parameter decoupling. It anatomizes the model into base and personalization subspaces, applying gradient sparsification to the shared base and pruning to the private personalization to decouple communication and computation costs from personalization quality. A rigorous convergence analysis reveals how sparsification and pruning jointly affect iteration complexity, leading to a joint optimization across per-client sparsity, pruning, and wireless bandwidth to minimize end-to-end training time. Empirical results on image and NLP tasks demonstrate faster convergence and substantial cost reductions with minimal accuracy loss, validating the practical value of coordinated resource-aware personalization in heterogeneous MEC environments.

Abstract

Federated learning enables collaborative machine learning while preserving data privacy, but high communication and computation costs, exacerbated by statistical and device heterogeneity, limit its practicality in mobile edge computing. Existing compression methods like sparsification and pruning reduce per-round costs but may increase training rounds and thus the total training cost, especially under heterogeneous environments. We propose a lightweight personalized FL framework built on parameter decoupling, which separates the model into shared and private subspaces, enabling us to uniquely apply gradient sparsification to the shared component and model pruning to the private one. This structural separation confines communication compression to global knowledge exchange and computation reduction to local personalization, protecting personalization quality while adapting to heterogeneous client resources. We theoretically analyze convergence under the combined effects of sparsification and pruning, revealing a sparsity-pruning trade-off that links to the iteration complexity. Guided by this analysis, we formulate a joint optimization that selects per-client sparsity and pruning rates and wireless bandwidth to reduce end-to-end training time. Simulation results demonstrate faster convergence and substantial reductions in overall communication and computation costs with negligible accuracy loss, validating the benefits of coordinated and resource-aware personalization in resource-constrained heterogeneous environments.

Paper Structure

This paper contains 27 sections, 5 theorems, 60 equations, 10 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and Motivations
  3. Related Works
  4. Compression techniques for reducing communication and computation costs
  5. Personalized Federated Learning
  6. Summary
  7. System Model
  8. Personalized Federated Learning Model
  9. Gradient Sparsification and Model Pruning
  10. Wireless Communication Model
  11. Convergence Analysis and Problem Formulation
  12. Convergence Analysis
  13. Problem Formulation
  14. Algorithm Design
  15. Extensions and Future Works
  16. ...and 12 more sections

Key Result

Lemma 1

For $\mathbf{x} \in \mathbb{R}^d$, $\mathbf{k} \in {\{ 0,1\}}^d$, where $\mathbf{k}$ contains $k$ elements equal to $1$, with $1\leq k\leq d$, according to the work NEURIPS2018_b440509a, it follows that Using the approach analogous to the proof of Eq. 36, Eq. 37 can be established.

Figures (10)

  • Figure 1: The workflow of our proposed approach.
  • Figure 2: Loss vs. Latency on MNIST for different non-IID cases. (a) Class-4. (b) Class-2. (c) $\alpha$ = 0.5. (d) $\alpha$ = 0.1.
  • Figure 3: Test Accuracy vs. Latency on MNIST for different non-IID cases. (a) Class-4. (b) Class-2. (c) $\alpha$ = 0.5. (d) $\alpha$ = 0.1.
  • Figure 4: Loss vs. Latency on CIFAR-10 for different non-IID cases. (a) Class-4. (b) Class-2. (c) $\alpha$ = 0.5. (d) $\alpha$ = 0.1.
  • Figure 5: Test Accuracy vs. Latency on CIFAR-10 for different non-IID cases. (a) Class-4. (b) Class-2. (c) $\alpha$ = 0.5. (d) $\alpha$ = 0.1.
  • ...and 5 more figures

Theorems & Definitions (18)

  • Lemma 1
  • proof
  • proof
  • Proposition 1
  • proof
  • Remark 1
  • Remark 2
  • Proposition 2
  • proof
  • Remark 3
  • ...and 8 more