Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Federated Low Rank Matrix Completion

Ahmed Ali Abbasi, Namrata Vaswani

TL;DR

The theoretical bounds on the sample complexity and iteration complexity of AltGDmin imply that it is the most communication-efficient solution while also been one of the most computation- and sample-efficient ones.

Abstract

In this work, we develop and analyze a Gradient Descent (GD) based solution, called Alternating GD and Minimization (AltGDmin), for efficiently solving the low rank matrix completion (LRMC) in a federated setting. LRMC involves recovering an $n \times q$ rank-$r$ matrix $\Xstar$ from a subset of its entries when $r \ll \min(n,q)$. Our theoretical guarantees (iteration and sample complexity bounds) imply that AltGDmin is the most communication-efficient solution in a federated setting, is one of the fastest, and has the second best sample complexity among all iterative solutions to LRMC. In addition, we also prove two important corollaries. (a) We provide a guarantee for AltGDmin for solving the noisy LRMC problem. (b) We show how our lemmas can be used to provide an improved sample complexity guarantee for AltMin, which is the fastest centralized solution.

Efficient Federated Low Rank Matrix Completion

TL;DR

The theoretical bounds on the sample complexity and iteration complexity of AltGDmin imply that it is the most communication-efficient solution while also been one of the most computation- and sample-efficient ones.

Abstract

In this work, we develop and analyze a Gradient Descent (GD) based solution, called Alternating GD and Minimization (AltGDmin), for efficiently solving the low rank matrix completion (LRMC) in a federated setting. LRMC involves recovering an rank- matrix from a subset of its entries when . Our theoretical guarantees (iteration and sample complexity bounds) imply that AltGDmin is the most communication-efficient solution in a federated setting, is one of the fastest, and has the second best sample complexity among all iterative solutions to LRMC. In addition, we also prove two important corollaries. (a) We provide a guarantee for AltGDmin for solving the noisy LRMC problem. (b) We show how our lemmas can be used to provide an improved sample complexity guarantee for AltMin, which is the fastest centralized solution.
Paper Structure (53 sections, 12 theorems, 84 equations, 1 figure, 1 table, 1 algorithm)

This paper contains 53 sections, 12 theorems, 84 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Setup and Notation
  3. Notation
  4. Assumption
  5. Federation
  6. Related Work
  7. Contributions and Novelty
  8. Proof Novelty
  9. Organization
  10. Alternating GD and Minimization (AltGDMin) Algorithm and Guarantees
  11. AltGDmin Algorithm
  12. Initialization
  13. AltGDmin iterations
  14. Federated AltGDmin
  15. AltGDmin Guarantee
  16. ...and 38 more sections

Key Result

Theorem 2.1

Pick an $\epsilon < 1$. Assume that Assumption incoh holds and that, entries of ${\bm{X}}^\star$ are observed independently with probability $p_{tot}$ satisfying $n q p_{tot} > C \kappa^6 \mu^2 q r^2 \log q \log({1}/{\epsilon})$. Set $\eta = 0.5/(p{\sigma_{\max}^{*2}})$ and $T = C\kappa^2 \log(1/\e (recall that ${\bm{X}}^{(T)} = {\bm{U}}^{(T)}\bm{B}^{(T)}$). The total per-node computation complex

Figures (1)

  • Figure 1: Figures (a),(c) compare federated implementations of AltGDmin (proposed), FactGD and AltMin. The results match what our theory (sufficient conditions) predicts. AltGDmin is the fastest due to its lowest communication-efficiency and due to all three having comparable computation cost. Figures (b), (d) compare iteration complexity numerically. The results match theory once again. Two versions of AltMin are compared: AltMin (Fed/Not-Prvt) uses exact LS solution for updating both $\bm{B}$ and ${\bm{U}}$. AltMin (Fed/Prvt) uses multiple gradient descent iterations to solve the LS problem for updating ${\bm{U}}$. In (a), we also compare FactGD with three choices of step size. See experiments' description for details.

Theorems & Definitions (19)

  • Theorem 2.1
  • Remark 2.2
  • Lemma 4.1: Initialization
  • Lemma 4.2: LS step analysis: error bound for $\bm{B}$
  • Lemma 4.3: Implications of error bound for $\bm{B}$
  • Lemma 4.4: Incoherence of $\bm{B}$
  • Lemma 4.5: Gradient expression and bounds
  • Lemma 4.6: Incoherence of ${\bm{U}}$
  • proof
  • Claim 4.7
  • ...and 9 more