Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Two-Timescale Approach for Wireless Federated Learning with Parameter Freezing and Power Control

Jinhao Ouyang, Yuan Liu, Hang Liu

TL;DR

The paper addresses energy efficiency in wireless federated learning by exploiting a two-timescale approach that freezes stabilized parameters on a slow frame-scale while adapting transmit power for unstable parameters on a fast slot-scale. It analyzes a bound on the convergence error that includes data heterogeneity, freezing-induced gaps, and outages, then formulates an online optimization problem $X_{av}$ to minimize the convergence error subject to per-device energy budgets $E_{av} \le \bar{E}_n$ via Lyapunov drift-plus-penalty. An online two-timescale algorithm decomposes into per-frame freezing decisions and per-slot power controls, with a threshold-based policy and a Majorization-Minimization solution to the non-convex freezing subproblem, achieving a convergence gap that scales as $O(1/V)$ while ensuring queue stability. Experiments on MNIST and CIFAR-10 demonstrate energy savings and superior convergence performance compared with heuristic schemes and state-of-the-art methods, highlighting practical impact for energy-constrained wireless FL.

Abstract

Federated learning (FL) enables distributed devices to train a shared machine learning (ML) model collaboratively while protecting their data privacy. However, the resource-limited mobile devices suffer from intensive computation-and-communication costs of model parameters. In this paper, we observe the phenomenon that the model parameters tend to be stabilized long before convergence during training process. Based on this observation, we propose a two-timescale FL framework by joint optimization of freezing stabilized parameters and controlling transmit power for the unstable parameters to balance the energy consumption and convergence. First, we analyze the impact of model parameter freezing and unreliable transmission on the convergence rate. Next, we formulate a two-timescale optimization problem of parameter freezing percentage and transmit power to minimize the model convergence error subject to the energy budget. To solve this problem, we decompose it into parallel sub-problems and decompose each sub-problem into two different timescales problems using the Lyapunov optimization method. The optimal parameter freezing and power control strategies are derived in an online fashion. Experimental results demonstrate the superiority of the proposed scheme compared with the benchmark schemes.

A Two-Timescale Approach for Wireless Federated Learning with Parameter Freezing and Power Control

TL;DR

The paper addresses energy efficiency in wireless federated learning by exploiting a two-timescale approach that freezes stabilized parameters on a slow frame-scale while adapting transmit power for unstable parameters on a fast slot-scale. It analyzes a bound on the convergence error that includes data heterogeneity, freezing-induced gaps, and outages, then formulates an online optimization problem to minimize the convergence error subject to per-device energy budgets via Lyapunov drift-plus-penalty. An online two-timescale algorithm decomposes into per-frame freezing decisions and per-slot power controls, with a threshold-based policy and a Majorization-Minimization solution to the non-convex freezing subproblem, achieving a convergence gap that scales as while ensuring queue stability. Experiments on MNIST and CIFAR-10 demonstrate energy savings and superior convergence performance compared with heuristic schemes and state-of-the-art methods, highlighting practical impact for energy-constrained wireless FL.

Abstract

Federated learning (FL) enables distributed devices to train a shared machine learning (ML) model collaboratively while protecting their data privacy. However, the resource-limited mobile devices suffer from intensive computation-and-communication costs of model parameters. In this paper, we observe the phenomenon that the model parameters tend to be stabilized long before convergence during training process. Based on this observation, we propose a two-timescale FL framework by joint optimization of freezing stabilized parameters and controlling transmit power for the unstable parameters to balance the energy consumption and convergence. First, we analyze the impact of model parameter freezing and unreliable transmission on the convergence rate. Next, we formulate a two-timescale optimization problem of parameter freezing percentage and transmit power to minimize the model convergence error subject to the energy budget. To solve this problem, we decompose it into parallel sub-problems and decompose each sub-problem into two different timescales problems using the Lyapunov optimization method. The optimal parameter freezing and power control strategies are derived in an online fashion. Experimental results demonstrate the superiority of the proposed scheme compared with the benchmark schemes.

Paper Structure

This paper contains 26 sections, 11 theorems, 75 equations, 12 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model
  3. Two-Timescale FL Structure
  4. Communication Model
  5. Computation Model
  6. Convergence Analysis And Problem Formulation
  7. Convergence Analysis
  8. Problem Formulation
  9. Problem Solution
  10. Problem Decomposition and Transformation
  11. Algorithm Design
  12. Algorithm Implementation
  13. Power Control Decision Per Slot
  14. Parameter Freezing Decision Per Frame
  15. Performance analysis
  16. ...and 11 more sections

Key Result

Lemma 1

For any model $\bm{w}_{n}^{t}, \tilde{\bm{w}}_{n}^{t} \in \mathbb{R}^D$, given a parameter freezing percentage $0 \leq \gamma \leq 1$, it holds that, where $\tilde{\bm{w}}_{n}^{t}$ represents the model updated with parameter freezing, $\bm{w}_{n}^{t}$ represents the local model without parameter freezing. Here, the expectation is taken over the randomness in the selection of frozen parameters.

Figures (12)

  • Figure 1: Percentage of stable parameters versus the training round index across different image classification tasks. In these experiments, a model parameter is considered stable if its cumulative absolute update over every consecutive $I=10$ updates is less than 0.5% of its total cumulative absolute update.
  • Figure 2: Two-timescale FL system.
  • Figure 3: Impact of control parameter $V$ on (a) MNIST dataset and (b) CIFAR-10 dataset.
  • Figure 4: Time evolution: (a) average energy consumption and (b) average cost on MNIST dataset.
  • Figure 5: Time evolution: (a) average energy consumption and (b) average cost on CIFAR-10 dataset.
  • ...and 7 more figures

Theorems & Definitions (23)

  • Definition 1
  • Lemma 1
  • proof
  • Definition 2
  • Theorem 1
  • proof
  • Corollary 1
  • Corollary 2
  • proof
  • Lemma 2
  • ...and 13 more