Generalized Lagrange Coded Computing: A Flexible Computation-Communication Tradeoff for Resilient, Secure, and Private Computation

TL;DR

Generalized Lagrange Coded Computing codes are proposed to simultaneously provide resiliency against stragglers who do not return computation results in time, security against adversarial workers who deliberately modify results for their benefit, and information-theoretic privacy of the dataset amidst possible collusion of workers.

Abstract

We consider the problem of evaluating arbitrary multivariate polynomials over a massive dataset containing multiple inputs, on a distributed computing system with a master node and multiple worker nodes. Generalized Lagrange Coded Computing (GLCC) codes are proposed to simultaneously provide resiliency against stragglers who do not return computation results in time, security against adversarial workers who deliberately modify results for their benefit, and information-theoretic privacy of the dataset amidst possible collusion of workers. GLCC codes are constructed by first partitioning the dataset into multiple groups, then encoding the dataset using carefully designed interpolating polynomials, and sharing multiple encoded data points to each worker, such that interference computation results across groups can be eliminated at the master. Particularly, GLCC codes include the state-of-the-art Lagrange Coded Computing (LCC) codes as a special case, and exhibit a more flexible tradeoff between communication and computation overheads in optimizing system efficiency. Furthermore, we apply GLCC to distributed training of machine learning models, and demonstrate that GLCC codes achieve a speedup of up to $2.5\text{--}3.9\times$ over LCC codes in training time, across experiments for training image classifiers on different datasets, model architectures, and straggler patterns.

Figures (5)

• Figure 1: A distributed computing system consisting of a master and $N$ workers for evaluating multivariate polynomials on dataset $X$. Worker $n$ computes a response $\widetilde{Y}_n$ to the master on local data $\widetilde{X}_n$. The master waits for the results from the fastest $K$ workers, $A$ out of whom may be malicious, to recover computation results. Up to $T$ workers may collude to infer about $X$.
• Figure 2: GLCC workflow.
• Figure 3: Average test accuracy of GLCC codes and LCC codes to train $M=5$ binary classifiers on MNIST and CIFAR-10 over training time. The cut-off time of horizontal coordinate is selected as the convergence time of GLCC codes with ($G=1,L=2$) for subfigures (a)-(b) and GLCC codes with ($G=5,L=1$) for subfigures (c)-(d).
• Figure 4: Running time of GLCC codes with ($G=5,L=1$) and LCC codes with respect to the number of colluding workers $T$. Here $T$ takes the values from $1$ to the maximum value before the privacy guarantee of LCC/GLCC codes is violated.
• Figure 5: Comparison of the average test accuracy of GLCC codes, LCC codes, and centralized training across iterations.

Theorems & Definitions (12)

• Theorem 1
• proof
• Remark 1
• Remark 2
• Remark 3
• Remark 4
• Remark 5
• Lemma 1: Generalized Cauchy Matrix Lin
• Lemma 2: Corollaries 10.8 and 10.12 in Von
• Lemma 3: Decoding Reed-Solomon Codes Gaochen2008complexityvon2013modernvan2022optimizing