Online computation of normalized substring complexity

TL;DR

This work addresses online computation of the compressibility measure $\delta$, defined as $\delta(w)=\max_{k\ge1}\{c_w[k]/k\}$ where $c_w[k]$ counts distinct length-$k$ substrings. It develops two online algorithms: an amortized $O(\log n)$ per-character method and a worst-case $O(\log^3 n)$ per-character method. The core approach sketches online updates to the substring-count array and reduces $\delta$ to a dynamic convex-hull problem, with $\delta$ equal to the maximal slope of a tangent from the origin to the hull. α-values and the hull are maintained via suffix-tree-like structures (CPA/BWT/LCS) and a Brewer-OVERMARS-inspired hull maintenance, enabling the first polylog-time online solution for this problem. The results have practical implications for streaming compressibility estimates related to LZ77/BWT-based methods and offer avenues for refined online string analyses.

Abstract

The normalized substring complexity $δ$ of a string is defined as $\max_k \{c[k]/k\}$, where $c[k]$ is the number of \textit{distinct} substrings of length $k$. This simply defined measure has recently attracted attention due to its established relationship to popular string compression algorithms. We consider the problem of computing $δ$ online, when the string is provided from a stream. We present two algorithms solving the problem: one working in $O(\log n)$ amortized time per character, and the other in $O(\log^3 n)$ worst-case time per character. To our knowledge, this is the first polylog-time online solution to this problem.

Figures (1)

• Figure 1: Point set $\mathcal{P}$ for $w=011010011001011010010110011010011$, its upper convex hull (green) and tangent line (red). Here $R=10$. Tangency point $(7,20)$ corresponds to $\delta(w)=20/7$.

Theorems & Definitions (23)

• lemma 1
• proof
• theorem 1: DBLP:journals/tcs/Luca99leve2001proof
• lemma 2
• proof
• lemma 3
• proof
• lemma 4
• proof
• theorem 2