Nearly Optimal Internal Dictionary Matching
Jingbang Chen, Jiangqi Dai, Qiuyang Mang, Qingyu Shi, Tingqiang Xu
TL;DR
The paper addresses internal dictionary matching (IDM) by introducing Basic Substring Structure (BASS), a linear-space framework that supports static dictionaries and enables near-optimal IDM queries on a text of length $n$ with dictionary size $d$. BASS organizes substrings via a grid and equivalence classes, leveraging PreTree and SufTree constructions to map blocks to efficient query structures, including a near-linear 2D range-counting backbone. The authors achieve improved time bounds across IDM queries: CountDistinct in $O(\log n)$ time with $O(n\log^2 n + d)$ space, Count in $O(\frac{\log n}{\log\log n})$ time with $O(n+d\sqrt{\log n})$ preprocessing, and ReportDistinct in $O(1+|output|)$ time with $O(n+d)$ preprocessing, while Exists/Report attain optimal $O(1+|output|)$ behavior. They also extend the approach to Range Longest Common Substring with constant-time queries and discuss potential applications to other internal query problems, highlighting BASS as a unifying and broadly applicable tool for IDM and related string-processing tasks.
Abstract
We study the internal dictionary matching (IDM) problem where a dictionary $\mathcal{D}$ containing $d$ substrings of a text $T$ is given, and each query concerns the occurrences of patterns in $\mathcal{D}$ in another substring of $T$. We propose a novel $O(n)$-sized data structure named Basic Substring Structure (BASS) where $n$ is the length of the text $T.$ With BASS, we are able to handle all types of queries in the IDM problem in nearly optimal query and preprocessing time. Specifically, our results include: $\bullet$ The first algorithm that answers the CountDistinct query in $\tilde{O}(1)$ time with $\tilde{O}(n+d)$ preprocessing, where we need to compute the number of distinct patterns that exist in $T[l,r]$. Previously, the best result was $\tilde{O}(m)$ time per query after $\tilde{O}(n^2/m+d)$ or $\tilde{O}(nd/m+d)$ preprocessing, where $m$ is a chosen parameter. $\bullet$ Faster algorithms for two other types of internal queries. We improve the runtime for (1) Occurrence counting (Count) queries to $O(\log n/\log\log n)$ time per query with $O(n+d\sqrt{\log n})$ preprocessing from $O(\log^2 n/\log\log n)$ time per query with $O(n\log n/\log \log n+d\log^{3/2} n)$ preprocessing. (2) Distinct pattern reporting (ReportDistinct) queries to $O(1+|\text{output}|)$ time per query from $O(\log n+|\text{output}|)$ per query. In addition, we match the optimal runtime in the remaining two types of queries, pattern existence (Exists), and occurrence reporting (Report). We also show that BASS is more generally applicable to other internal query problems.