Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

General Coded Computing: Adversarial Settings

Parsa Moradi, Hanzaleh Akbarinodehi, Mohammad Ali Maddah-Ali

TL;DR

General Coded Computing addresses robustness of distributed computations beyond structured tasks by constraining encoder/decoder to second-order Sobolev spaces and employing smoothing-spline–based kernel representations. It shows that with at most $\gamma=\mathcal{O}(N^{a})$ adversaries, the average error decays at rate $\mathcal{O}(N^{\tfrac{6}{5}(a-1)})$, and proves optimal robustness in terms of the maximum tolerable adversaries. The work provides a concrete, kernel-based encoder/decoder design with theoretical guarantees and demonstrates practical effectiveness on function evaluation and deep neural-network inference. These results establish a principled foundation for reliable general coded computing in adversarial environments, balancing approximation, regularization, and adversarial resilience.

Abstract

Conventional coded computing frameworks are predominantly tailored for structured computations, such as matrix multiplication and polynomial evaluation. Such tasks allow the reuse of tools and techniques from algebraic coding theory to improve the reliability of distributed systems in the presence of stragglers and adversarial servers. This paper lays the foundation for general coded computing, which extends the applicability of coded computing to handle a wide class of computations. In addition, it particularly addresses the challenging problem of managing adversarial servers. We demonstrate that, in the proposed scheme, for a system with $N$ servers, where $\mathcal{O}(N^a)$, $a \in [0,1)$, are adversarial, the supremum of the average approximation error over all adversarial strategies decays at a rate of $N^{\frac{6}{5}(a-1)}$, under minimal assumptions on the computing tasks. Furthermore, we show that within a general framework, the proposed scheme achieves optimal adversarial robustness, in terms of maximum number of adversarial servers it can tolerate. This marks a significant step toward practical and reliable general coded computing. Implementation results further validate the effectiveness of the proposed method in handling various computations, including inference in deep neural networks.

General Coded Computing: Adversarial Settings

TL;DR

General Coded Computing addresses robustness of distributed computations beyond structured tasks by constraining encoder/decoder to second-order Sobolev spaces and employing smoothing-spline–based kernel representations. It shows that with at most adversaries, the average error decays at rate , and proves optimal robustness in terms of the maximum tolerable adversaries. The work provides a concrete, kernel-based encoder/decoder design with theoretical guarantees and demonstrates practical effectiveness on function evaluation and deep neural-network inference. These results establish a principled foundation for reliable general coded computing in adversarial environments, balancing approximation, regularization, and adversarial resilience.

Abstract

Conventional coded computing frameworks are predominantly tailored for structured computations, such as matrix multiplication and polynomial evaluation. Such tasks allow the reuse of tools and techniques from algebraic coding theory to improve the reliability of distributed systems in the presence of stragglers and adversarial servers. This paper lays the foundation for general coded computing, which extends the applicability of coded computing to handle a wide class of computations. In addition, it particularly addresses the challenging problem of managing adversarial servers. We demonstrate that, in the proposed scheme, for a system with servers, where , , are adversarial, the supremum of the average approximation error over all adversarial strategies decays at a rate of , under minimal assumptions on the computing tasks. Furthermore, we show that within a general framework, the proposed scheme achieves optimal adversarial robustness, in terms of maximum number of adversarial servers it can tolerate. This marks a significant step toward practical and reliable general coded computing. Implementation results further validate the effectiveness of the proposed method in handling various computations, including inference in deep neural networks.

Paper Structure

This paper contains 14 sections, 23 theorems, 84 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Proposed Scheme
  4. Decoder Design
  5. Encoder Design
  6. Theoretical Guarantees
  7. Experimental Results
  8. Proof Sketch
  9. Preliminaries: Sobolev spaces and Sobolev norms
  10. Smoothing Splines
  11. Proof of Theorem \ref{['th:dec_bound']}
  12. Proof of Theorem \ref{['th:enc_bound']}
  13. Proof of Corollary \ref{['cor:conv_rate']}
  14. Proof of Theorem \ref{['th:letcc_enc_ss']}

Key Result

Theorem 1

(Impossibility Result) In the proposed coded computing framework, assume $\gamma = \mu N$ for some $\mu \in (0, 1)$. Then, there exists some $f$ with bounded first and second derivative, for which there is no $u_\textrm{e}, u_\textrm{d} \in \mathcal{H}^{2}\left(\Omega\right)$ such that: $\lim_{N \t

Figures (1)

  • Figure 1: Log-log plot illustrating the convergence rates of approximation error for the function $f(x) = x\sin(x)$ and LeNet5 network under various number of adversarial worker nodes.

Theorems & Definitions (32)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Corollary 1
  • Remark 1
  • Theorem 4
  • Lemma 1
  • Definition 1: Sobolev Space
  • Definition 2
  • Theorem 5: Theorem 7.34, leoni2024first
  • ...and 22 more