Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coded Data Rebalancing for Distributed Data Storage Systems with Cyclic Storage

Abhinav Vaishya, Athreya Chandramouli, Srikar Kale, Prasad Krishnan

TL;DR

This work studies coded data rebalancing for cyclic, balanced distributed databases, addressing node removal and addition with a focus on practical file sizes. It proves the existence of two coded rebalancing schemes for node removal and an optimal uncoded scheme for node addition, achieving substantially lower communication loads than uncoded baselines while requiring only cubic file size in the number of nodes. A specific data-placement theorems are established, including a lower bound for the node-removal case and a multiplicative gap guarantee (up to a factor of 13) between the achievable scheme and this bound. The results demonstrate that cyclic placement enables scalable, efficient rebalancing and offer concrete transmission schemes, analysis of loads, and numerical comparisons to validate practical benefits in distributed storage systems.

Abstract

We consider replication-based distributed storage systems in which each node stores the same quantum of data and each data bit stored has the same replication factor across the nodes. Such systems are referred to as balanced distributed databases. When existing nodes leave or new nodes are added to this system, the balanced nature of the database is lost, either due to the reduction in the replication factor, or the non-uniformity of the storage at the nodes. This triggers a rebalancing algorithm, that exchanges data between the nodes so that the balance of the database is reinstated. The goal is then to design rebalancing schemes with minimal communication load. In a recent work by Krishnan et al., coded transmissions were used to rebalance a carefully designed distributed database from a node removal or addition. These coded rebalancing schemes have optimal communication load, however, require the file-size to be at least exponential in the system parameters. In this work, we consider a cyclic balanced database (where data is cyclically placed in the system nodes) and present coded rebalancing schemes for node removal and addition in such a database. These databases (and the associated rebalancing schemes) require the file-size to be only cubic in the number of nodes in the system. We bound the advantage of our node removal rebalancing scheme over the uncoded scheme, and show that our scheme has a smaller communication load. In the node addition scenario, the rebalancing scheme presented is a simple uncoded scheme, which we show has optimal load. Finally, we derive a lower bound for the single node-removal rebalancing for the specific choice of data placements specified by our achievable rebalancing schemes, and show that our achievable rebalancing loads are within a multiplicative gap from the lower bound obtained.

Coded Data Rebalancing for Distributed Data Storage Systems with Cyclic Storage

TL;DR

This work studies coded data rebalancing for cyclic, balanced distributed databases, addressing node removal and addition with a focus on practical file sizes. It proves the existence of two coded rebalancing schemes for node removal and an optimal uncoded scheme for node addition, achieving substantially lower communication loads than uncoded baselines while requiring only cubic file size in the number of nodes. A specific data-placement theorems are established, including a lower bound for the node-removal case and a multiplicative gap guarantee (up to a factor of 13) between the achievable scheme and this bound. The results demonstrate that cyclic placement enables scalable, efficient rebalancing and offer concrete transmission schemes, analysis of loads, and numerical comparisons to validate practical benefits in distributed storage systems.

Abstract

We consider replication-based distributed storage systems in which each node stores the same quantum of data and each data bit stored has the same replication factor across the nodes. Such systems are referred to as balanced distributed databases. When existing nodes leave or new nodes are added to this system, the balanced nature of the database is lost, either due to the reduction in the replication factor, or the non-uniformity of the storage at the nodes. This triggers a rebalancing algorithm, that exchanges data between the nodes so that the balance of the database is reinstated. The goal is then to design rebalancing schemes with minimal communication load. In a recent work by Krishnan et al., coded transmissions were used to rebalance a carefully designed distributed database from a node removal or addition. These coded rebalancing schemes have optimal communication load, however, require the file-size to be at least exponential in the system parameters. In this work, we consider a cyclic balanced database (where data is cyclically placed in the system nodes) and present coded rebalancing schemes for node removal and addition in such a database. These databases (and the associated rebalancing schemes) require the file-size to be only cubic in the number of nodes in the system. We bound the advantage of our node removal rebalancing scheme over the uncoded scheme, and show that our scheme has a smaller communication load. In the node addition scenario, the rebalancing scheme presented is a simple uncoded scheme, which we show has optimal load. Finally, we derive a lower bound for the single node-removal rebalancing for the specific choice of data placements specified by our achievable rebalancing schemes, and show that our achievable rebalancing loads are within a multiplicative gap from the lower bound obtained.
Paper Structure (21 sections, 5 theorems, 20 equations, 12 figures, 7 algorithms)

This paper contains 21 sections, 5 theorems, 20 equations, 12 figures, 7 algorithms.

Table of Contents

  1. Introduction
  2. Rebalancing Schemes for Cyclic Databases
  3. Proof of Theorem \ref{['thm:achieve']}: Rebalancing for Single Node Removal In Cyclic Databases
  4. Intuition for the Rebalancing Scheme
  5. Examples
  6. Algorithm for Rebalancing for Node-Removal
  7. Correctness of Algorithm \ref{['algo:rebalancing']}
  8. Communication Load of Algorithm \ref{['algo:rebalancing']}
  9. Scheme 1
  10. Scheme 2
  11. Advantage of Algorithm \ref{['algo:rebalancing']} over the uncoded scheme
  12. Proof of Theorem \ref{['thm:achieve']}: Rebalancing for single node addition in cyclic databases
  13. Correctness of Algorithm \ref{['algo:nodeaddition']}
  14. Communication Load of Algorithm \ref{['algo:nodeaddition']}
  15. Lower Bound for Rebalancing Load for Node-Removal
  16. ...and 6 more sections

Key Result

Theorem 1

For an $r$-balanced cyclic database having $K$ nodes and $r\in \{3,\hdots,K-1\}$, if the segment size $T$ is divisible by $2(K^2-1)$, then rebalancing schemes for node removal and addition exist which achieve the respective communication loads where, $L_1(r) = \frac{(K-r)(2r-1)}{(K-1)}$ and $L_2(r) = \frac{1}{2(K-1)}\left(K(r-1)+ \lceil \frac{r^2-2r}{2} \rceil\right)$. Also, the following relatio

Figures (12)

  • Figure 1: $r$-balanced cyclic database on nodes $[K]$
  • Figure 2: For $K=15$, the figure shows comparisons of communication loads of Scheme 1 and Scheme 2 with the load of uncoded transmission scheme, for varying $r$. Note that all curves are relevant only for $r\in\{3,\hdots,K-1\}$. We see that the minimum of the loads of the two schemes is always less than the uncoded load. Further, by Claim \ref{['claim:threshold']}, any integer value of $r\geq 11$, we see that Scheme 1 has smaller load than Scheme 2, and the reverse is true otherwise.
  • Figure 3: Target cyclic database on nodes $[K-1]$
  • Figure 4: The matrix $M$ for $K=8$, $r = 6$. The rows correspond to subsegments and the columns correspond to nodes. Entry $M_{i,j}$ = '$*$' if the $i^{\text{th}}$ subsegment is contained in the $j^{\text{th}}$ node. $M_{i,j}$ = '$s$' if the $i^{\text{th}}$ subsegment must be delivered to the $j^{\text{th}}$ node. For each shape enclosing an entry, the row and column corresponding each entry with that shape gives a valid XOR-coded transmission.
  • Figure 5: Matrix $M$ for $K=6, r=3$ case. The rows of this matrix $M$ correspond to subsegments and the columns correspond to nodes. Entry $M_{i,j}$ = '$*$' if the $i^{\text{th}}$ subsegment is contained in the $j^{\text{th}}$ node. $M_{i,j}$ = '$s$' if the $i^{\text{th}}$ subsegment must be delivered to the $j^{\text{th}}$ node. For each shape enclosing an entry, the row and column corresponding each entry with that shape lead to a XOR-coded transmission.
  • ...and 7 more figures

Theorems & Definitions (14)

  • Definition 1
  • Theorem 1
  • Remark 1
  • Claim 1
  • Remark 2
  • Remark 3
  • Lemma 1
  • Example 1
  • Claim 2
  • Lemma 2
  • ...and 4 more