On Optimal Batch Size in Coded Computing

TL;DR

This work analyzes how batching interacts with erasure coding in coded computing to minimize expected job completion time under parallel workers. By modeling the service time with shifted exponential distributions and deriving large-$n$ asymptotics, it shows that the optimal batch size $b^*$ tends to extreme values (either $1$ or $s$) for a fixed code rate $R$, with a critical threshold $R' \approx 0.72$ separating regimes where replication or full-batch coding is preferable. It further develops a joint optimization framework over $(b,R)$, revealing that the best strategy depends on the straggling characteristics and job scale, and provides both analytical results and simulations to guide practical system design. Collectively, the results offer actionable guidance for selecting batch size and redundancy in distributed coded computing to reduce expected completion times under varying service-time distributions and system parameters.

Abstract

We consider computing systems that partition jobs into tasks, add redundancy through coding, and assign the encoded tasks to different computing nodes for parallel execution. The expected execution time depends on the level of redundancy. The computing nodes execute large jobs in batches of tasks. We show that the expected execution time depends on the batch size as well. The optimal batch size that minimizes the execution time depends on the level of redundancy under a fixed number of parallel servers and other system parameters. Furthermore, we show how to (jointly) optimize the redundancy level and batch size to reduce the expected job completion time for two service-time distributions. The simulation presented helps us appreciate the claims.

Figures (3)

• Figure 1: Distributed Computing System. Here, job size $J=6$ CUs is replicated, and $3$ batch generations are formed for selecting $b=2$ CUs. The ongoing batch generation $G_1$ gets completed when any $1$ worker finishes the batch task. The job will be completed when the $G_3$ batch generation is executed.
• Figure 2: Optimizing batch size for different redundancy levels. (a) The optimum batch size is the maximum for redundancy level $R=0.8$ and is the minimum for redundancy level $R=0.4$. For (b) and (c), we have the same redundancy level, $R=0.7$, but different job sizes. (b) is optimum with maximum batch size. (c) is optimum with minimum batch size.
• Figure 3: Optimizing the expected job completion time: (a) and (b) have the same job size and number of workers. The optimal strategy is replication with minimum batch size for $W=1,\Updelta=0.1$ and splitting with minimum batch size when $W=1,\Updelta=10$. Cases (c) and (d) have $W=\Updelta=1$ and differ by the job scale factor. Coding with the minimum batch size is optimal for the low job scale factor, and splitting with the maximum batch size is optimum for the high job scale factor.

Theorems & Definitions (11)

• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3
• proof
• Theorem 4
• Theorem 5
• proof
• Theorem 6