Randomized Kaczmarz in Adversarial Distributed Setting

TL;DR

The efficiency of the proposed iterative approach in the presence of adversaries and its ability to identify adversarial workers with high accuracy and tolerate varying levels of adversary rates are demonstrated.

Abstract

Developing large-scale distributed methods that are robust to the presence of adversarial or corrupted workers is an important part of making such methods practical for real-world problems. In this paper, we propose an iterative approach that is adversary-tolerant for convex optimization problems. By leveraging simple statistics, our method ensures convergence and is capable of adapting to adversarial distributions. Additionally, the efficiency of the proposed methods for solving convex problems is shown in simulations with the presence of adversaries. Through simulations, we demonstrate the efficiency of our approach in the presence of adversaries and its ability to identify adversarial workers with high accuracy and tolerate varying levels of adversary rates.

Figures (8)

• Figure 1: Effects of the number of used rows $d_0$ on convergence: $N_r=20, n_r=4, k=3, \|e\|_{\infty} = 10^{-3}$. The error norms were averaged over $50$ trials (the solid lines) with $90\%$ percentiles (the shaded areas).
• Figure 1: Effects of the number of used rows $d_0$ on convergence: $p=0.4$, $\|e\|_{\infty} = 1$, $N_r = 20$, $n_r = 15$, and $\gamma=1$. The object function $F(x) = \sum_{i=1}^{d_1} \left(\frac{1}{2}(A_ix_j - b_i)^2 + \frac{\gamma}{d_1} \|x\|_1 \right)$.
• Figure 2: Effects of different data sizes $d_0$ on convergence: $N_r=20, n_r=4, k=3$, and $\|e\|_{\infty} = 500$. The error norms were averaged over $50$ trials (the solid lines) with $90\%$ percentiles (the shaded areas).
• Figure 2: Effects of the number of used rows $d_0$ on convergence: $p=0.6$, $\|e\|_{\infty} = 10^{-3}$, $N_r = 20$, $n_r = 4$, $r = 3$, and $\alpha=0.7$ using RrDR method.
• Figure 3: Effects of the number of used workers $n_r$ for row $r$ (note that $n_r \equiv n$ for all selected rows): $k=3$, $d_0=6$, $N_r=20$, $\|e\|_{\infty}=500$.

Theorems & Definitions (20)

• Lemma 3.1
• Proof 1
• Lemma 3.2
• Lemma 3.3
• Proof 2
• Corollary 3.4
• Theorem 3.5
• Lemma 3.6
• Proof 3
• Lemma 3.7