Dynamic r-index: An Updatable Self-Index in LCP-bounded Time

TL;DR

The paper tackles dynamic string indexing for highly repetitive data by introducing the dynamic r-index, an updatable self-index based on run-length BWT. It extends the static r-index with three coordinated dynamic data structures and a dynamic LF mapping to support efficient insertions and deletions while preserving fast count and locate queries. The authors provide detailed algorithms, complexity analyses, and extensive experiments showing LCP-bounded update times and practical performance on large, repetitive datasets. This work enables real-time maintenance of compressed self-indexes in evolving text collections, with substantial space efficiency on repetitive data and open-source release plans. Overall, the dynamic r-index offers a principled, scalable approach to updatable self-indexing for modern repetitive workloads.

Abstract

A self-index is a compressed data structure that supports locate queries -- reporting all positions where a given pattern occurs in a string while maintaining the string in compressed form. While many self-indexes have been proposed, developing dynamically updatable ones supporting string insertions and deletions remains a challenge. The r-index (Gagie et al., JACM'20) is a representative static self-index based on the run-length Burrows-Wheeler transform (RLBWT), designed for highly repetitive strings. We present the dynamic r-index, a dynamic extension of the r-index that achieves updates in LCP-bounded time. The dynamic r-index supports count queries in $O(m \log r / \log \log r)$ time and locate queries in $O(m \log r / \log \log r + \mathsf{occ} \log r)$ time, using $O(r)$ words of space, where $m$ is the length of a query with $\mathsf{occ}$ occurrences and $r$ is the number of runs in the RLBWT. Crucially, update operations are supported in $O((m + L_{\mathsf{max}}) \log n)$ time for a substring of length $m$, where $L_{\mathsf{max}}$ is the maximum LCP value; the average running time is $O((m + L_{\mathsf{avg}}) \log n)$, where $L_{\mathsf{avg}}$ is the average LCP value. This LCP-bounded complexity is particularly advantageous for highly repetitive strings where LCP values are typically small. We experimentally demonstrate the practical efficiency of the dynamic r-index on various highly repetitive datasets.

Figures (9)

• Figure 1: Table illustrating the suffix array ($\mathsf{SA}$), inverse suffix array ($\mathsf{ISA}$), LCP array ($\mathsf{LCP}$), conceptual matrix ($\mathsf{CM}$), and BWT ($L$) of $T = \texttt{bbabba\$}$.
• Figure 2: (Left) Table illustrating the suffix array ($\mathsf{SA}$), conceptual matrix ($\mathsf{CM}$), and BWT ($L$) of the string $T$ used in Figure \ref{['fig:bwt']}. The yellow rectangle represents the sa-interval $[sp_{1}, ep_{1}]$ of $\texttt{b}$. Similarly, the gray rectangle represents the sa-interval $[sp_{2}, ep_{2}]$ of $\texttt{ab}$. (Right) The sampled suffix arrays $\mathsf{SA}_{s}$ and $\mathsf{SA}_{e}$ for $\mathsf{SA}$.
• Figure 3: Illustration of the update process from $\mathsf{CM}$, $L_{\mathsf{RLE}}$, $\mathsf{SA}_{s}$, and $\mathsf{SA}_{e}$ to $\mathsf{CM}'$, $L'_{\mathsf{RLE}}$, $\mathsf{SA}'_{s}$, and $\mathsf{SA}'_{e}$, respectively. Here, $T = \texttt{bbabba\$}$ and $T' = \texttt{bbabbba\$}$, where $T'$ is obtained by inserting the character $b$ into $T$ at position 6. In each $\mathsf{CM}_j$, the circular shift to be removed is highlighted in gray, while the newly inserted circular shift is highlighted in yellow. The BWT corresponding to $\mathsf{CM}_{j}$ is shown with bold characters. See Appendix \ref{['sec:appendix']} for the full version of this figure.
• Figure 4: Illustration of the dynamic LF function for the example shown in Figure \ref{['fig:update_CM']}. Here, $L_{6} = \texttt{abbbba\$}$, $L_{5} = \texttt{abbbbba\$}$, $\mathsf{SA}_{6} = [8, 7, 3, 5, 2, 4, 1]$, and $\mathsf{SA}_{5} = [8, 7, 3, 5, 6, 2, 4, 1]$. The arrow starting at each $t$-th row of $\mathsf{CM}_{5}$ represents $\mathsf{LF}_{5}(t) = t'$, where $t'$ is the index of the row pointed to by the arrow.
• Figure 5: Illustration of the relationship between the two suffixes $T[u..n]$ and $T[\mathsf{SA}_{s}[\lambda]..n]$ stated in Lemma \ref{['lem:lambda_prop']}.

Theorems & Definitions (30)

• Lemma 1: 10.1145/3375890
• Lemma 2: Section 2.3 in DBLP:journals/tcs/BannaiGI20
• Corollary 3
• Lemma 4
• Lemma 5
• Lemma 6
• Lemma 7: Dynamic LF formula
• Lemma 8
• Lemma 9
• Lemma 10