Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robustness of Iteratively Pre-Conditioned Gradient-Descent Method: The Case of Distributed Linear Regression Problem

Kushal Chakrabarti, Nirupam Gupta, Nikhil Chopra

TL;DR

This letter considers the problem of multi-agent distributed linear regression in the presence of system noises, and empirically shows that the robustness of the IPG method compares favorably to the state-of-the-art algorithms.

Abstract

This paper considers the problem of multi-agent distributed linear regression in the presence of system noises. In this problem, the system comprises multiple agents wherein each agent locally observes a set of data points, and the agents' goal is to compute a linear model that best fits the collective data points observed by all the agents. We consider a server-based distributed architecture where the agents interact with a common server to solve the problem; however, the server cannot access the agents' data points. We consider a practical scenario wherein the system either has observation noise, i.e., the data points observed by the agents are corrupted, or has process noise, i.e., the computations performed by the server and the agents are corrupted. In noise-free systems, the recently proposed distributed linear regression algorithm, named the Iteratively Pre-conditioned Gradient-descent (IPG) method, has been claimed to converge faster than related methods. In this paper, we study the robustness of the IPG method, against both the observation noise and the process noise. We empirically show that the robustness of the IPG method compares favorably to the state-of-the-art algorithms.

Robustness of Iteratively Pre-Conditioned Gradient-Descent Method: The Case of Distributed Linear Regression Problem

TL;DR

This letter considers the problem of multi-agent distributed linear regression in the presence of system noises, and empirically shows that the robustness of the IPG method compares favorably to the state-of-the-art algorithms.

Abstract

This paper considers the problem of multi-agent distributed linear regression in the presence of system noises. In this problem, the system comprises multiple agents wherein each agent locally observes a set of data points, and the agents' goal is to compute a linear model that best fits the collective data points observed by all the agents. We consider a server-based distributed architecture where the agents interact with a common server to solve the problem; however, the server cannot access the agents' data points. We consider a practical scenario wherein the system either has observation noise, i.e., the data points observed by the agents are corrupted, or has process noise, i.e., the computations performed by the server and the agents are corrupted. In noise-free systems, the recently proposed distributed linear regression algorithm, named the Iteratively Pre-conditioned Gradient-descent (IPG) method, has been claimed to converge faster than related methods. In this paper, we study the robustness of the IPG method, against both the observation noise and the process noise. We empirically show that the robustness of the IPG method compares favorably to the state-of-the-art algorithms.

Paper Structure

This paper contains 11 sections, 80 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Summary of Our Contributions
  3. ROBUSTNESS OF THE IPG METHOD
  4. Notation, Assumption and Prior Results
  5. Robustness against Observation Noise
  6. Robustness against Process Noise
  7. PROOFS OF THEOREMS \ref{['thm:meas_noise']} AND \ref{['thm:sys_noise']}
  8. Proof of Theorem \ref{['thm:meas_noise']}
  9. Proof of Theorem \ref{['thm:sys_noise']}
  10. EXPERIMENTAL RESULTS
  11. SUMMARY

Figures (2)

  • Figure 1: System architecture.
  • Figure 2: Temporal evolution of estimation error $\left\lVert x(t)-x^*\right\rVert$ in presence of additive observation noise (shown in the first row) and process noise (shown in the second row), for the different algorithms represented by different colors. For algorithms IPG, GD, NAG, HBM, and BFGS, $x(0) = [0,\ldots,0]^T$. Additionally, for IPG, $K(0) = O_{d \times d}$, and for BFGS, $M(0) = I$ the identity matrix. APC is initialized as per the instructions in azizan2019distributed. The other parameters are enlisted in Table \ref{['tab:parameters']}.