New Wide Locally Recoverable Codes with Unified Locality

TL;DR

Wide Locally Recoverable Codes (LRCs) aim for ultra-high code rates but face reliability-performance tradeoffs during common events. The paper identifies three locality limitations—recovery locality, topology locality, and XOR locality—and introduces UniLRC to unify these aspects in the code design. UniLRC uses a Vandermonde-based generator matrix combined with matrix decompositions to tightly couple local and global parity, achieving distance-optimal fault tolerance while optimizing locality. System-level and theoretical evaluations show significant gains in normal read throughput, degraded read latency, and recovery throughput with reduced cross-cluster traffic, indicating strong practical impact for large-scale distributed storage systems.

Abstract

Wide Locally Recoverable Codes (LRCs) have recently been proposed as a solution for achieving high reliability, good performance, and ultra-low storage cost in distributed storage systems. However, existing wide LRCs struggle to balance optimal fault tolerance and high availability during frequent system events. By analyzing the existing LRCs, we reveal three limitations in the LRC construction which lay behind the non-optimal overall performance from multiple perspectives, including non-minimum local recovery cost, non cluster-topology-aware data distribution, and non XOR-based local coding. Thanks to the flexible design space offered by the locality property of wide LRCs, we present UniLRC, which unifies locality considerations in code construction. UniLRC achieves the optimal fault tolerance while overcoming the revealed limitations. We implement UniLRC prototype and conduct comprehensive theoretical and system evaluations, showing significant improvements in reliability and performance over existing wide LRCs deployed in Google and Azure clusters.

Figures (12)

• Figure 1: Different wide LRCs with $n=42, k=30$. The LRCs are from Microsoft huang2012erasure (ALRC) and Google kadekodi2023practical (OLRC and ULRC). The terms $d_i$, $l_i$ and $g_i$ mean data block, local and global parity block, respectively.
• Figure 2: Topology locality of ECWide for ULRC$(42,30,\{7,8\})$.
• Figure 3: Comparisons of XOR and MUL under coding computing. (a) Coding throughput on XOR and MUL. (b) The average times of XOR and MUL for decoding a block with baseline LRCs.
• Figure 4: An example of the UniLRC$(42,30,6)$. A local group is assigned to a single cluster, with all data blocks, as well as local and global parity blocks, uniformly distributed across clusters. Local parity blocks are generated using XOR. This configuration tolerates up to any $g+1 = 7$ block failures and one cluster failure.
• Figure 5: Trade-off on cluster number, scale coefficient, code rate and stripe width for UniLRC with $z \leq 20$ and $\alpha = 1,2,3$.

Theorems & Definitions (7)

• definition 1: Linear code huffman2010fundamentals
• definition 2: $(n, k, r)$-LRC asteris2013xoringhuang2012erasure
• theorem 1: Singleton bound and distance optimal LRCsgopalan2012localitypapailiopoulos2012locallyxing2022constructionguruswami2018long
• theorem 2
• theorem 3
• theorem 4
• theorem 5