Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Logarithmically Quantized Distributed Optimization over Dynamic Multi-Agent Networks

Mohammadreza Doostmohammadian, Sérgio Pequito

TL;DR

The proposed optimization dynamics comprise a primary state variable converging to the optimizer and an auxiliary variable tracking the objective function’s gradient, resulting in a hybrid system requiring convergence analysis using matrix perturbation theory and eigenspectrum analysis.

Abstract

Distributed optimization finds many applications in machine learning, signal processing, and control systems. In these real-world applications, the constraints of communication networks, particularly limited bandwidth, necessitate implementing quantization techniques. In this paper, we propose distributed optimization dynamics over multi-agent networks subject to logarithmically quantized data transmission. Under this condition, data exchange benefits from representing smaller values with more bits and larger values with fewer bits. As compared to uniform quantization, this allows for higher precision in representing near-optimal values and more accuracy of the distributed optimization algorithm. The proposed optimization dynamics comprise a primary state variable converging to the optimizer and an auxiliary variable tracking the objective function's gradient. Our setting accommodates dynamic network topologies, resulting in a hybrid system requiring convergence analysis using matrix perturbation theory and eigenspectrum analysis.

Logarithmically Quantized Distributed Optimization over Dynamic Multi-Agent Networks

TL;DR

The proposed optimization dynamics comprise a primary state variable converging to the optimizer and an auxiliary variable tracking the objective function’s gradient, resulting in a hybrid system requiring convergence analysis using matrix perturbation theory and eigenspectrum analysis.

Abstract

Distributed optimization finds many applications in machine learning, signal processing, and control systems. In these real-world applications, the constraints of communication networks, particularly limited bandwidth, necessitate implementing quantization techniques. In this paper, we propose distributed optimization dynamics over multi-agent networks subject to logarithmically quantized data transmission. Under this condition, data exchange benefits from representing smaller values with more bits and larger values with fewer bits. As compared to uniform quantization, this allows for higher precision in representing near-optimal values and more accuracy of the distributed optimization algorithm. The proposed optimization dynamics comprise a primary state variable converging to the optimizer and an auxiliary variable tracking the objective function's gradient. Our setting accommodates dynamic network topologies, resulting in a hybrid system requiring convergence analysis using matrix perturbation theory and eigenspectrum analysis.

Paper Structure

This paper contains 7 sections, 2 theorems, 25 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Problem Formulation and Assumptions
  3. Logarithmically Quantized Distributed Optimization over Dynamic Multi-Agent Networks
  4. The Proposed Distributed Log-Quantized Dynamics
  5. Convergence of Distributed Log-Quantized Dynamics
  6. Simulation: Application to SVM
  7. Conclusion and Future Research

Key Result

Lemma 1

The proposed dynamics eq_xydot1-eq_M_g can be written as the following perturbation-based form, where with $M^0 = \left( \right)$, and $M^1 = \left( \right)$, where the following linear matrix inequalities hold:

Figures (4)

  • Figure 1: The time-evolution of the classifying line parameters $\boldsymbol{\omega}_i \in \mathbb R^2$ and $\nu_i \in \mathbb R$ are shown in this figure. As it is clear from the figure, all agents have reached a consensus over these parameters. The minor oscillations in $\nu_i$ states are due to changes in the network topology.
  • Figure 2: The separating SVM lines by different agents at time $t=1.25$ sec (Left) and time $t=2.5$ sec (Right).
  • Figure 3: (Left) Comparison between different distributed optimization solutions with our proposed quantized setup, (Right) Comparison of the residual convergence for different logarithmic quantization levels $\rho$.
  • Figure 4: Residual under uniform vs. log-scale quantization: Clearly, optimization under uniform quantization results in large optimality gap.

Theorems & Definitions (4)

  • Lemma 1
  • proof
  • Theorem 1
  • proof