Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Exact String Matching via Statistical Anchoring

Omar Garraoui

TL;DR

This work tackles exact string matching in natural language text and aims to reduce character comparisons by reordering verification. It introduces a preprocessing step that identifies a statistically rare anchor in the pattern using a frequency table $freq(c)$ and then verifies at $a = \arg\min_{i \in [0, m-1]} freq(P[i])$ before other positions, while preserving the Horspool shift heuristic. The algorithm maintains $O(m)$ preprocessing and $O(n/m)$ average-case search, and in experiments on Italian text yields $5.33\%$ fewer comparisons than standard BMH (303{,}194 vs 320{,}275) with a $100\%$ win rate across $12$ patterns; gains are most pronounced for patterns containing rare characters. The approach is simple to integrate, scalable to large corpora, and demonstrates that domain-specific statistics can yield meaningful practical speedups without increasing asymptotic complexity.

Abstract

In this work, we propose an enhancement to the Boyer-Moore-Horspool algorithm tailored for natural language text. The approach involves preprocessing the search pattern to identify its statistically least frequent character, referred to as the "anchor." During the search, verification is first performed at this high-entropy position, allowing the algorithm to quickly discard non-matching windows. This fail-fast strategy reduces unnecessary comparisons, improving overall efficiency. Our implementation shows that incorporating basic linguistic statistics into classical pattern-matching techniques can boost performance without increasing complexity to the shift heuristics.

Optimizing Exact String Matching via Statistical Anchoring

TL;DR

This work tackles exact string matching in natural language text and aims to reduce character comparisons by reordering verification. It introduces a preprocessing step that identifies a statistically rare anchor in the pattern using a frequency table and then verifies at before other positions, while preserving the Horspool shift heuristic. The algorithm maintains preprocessing and average-case search, and in experiments on Italian text yields fewer comparisons than standard BMH (303{,}194 vs 320{,}275) with a win rate across patterns; gains are most pronounced for patterns containing rare characters. The approach is simple to integrate, scalable to large corpora, and demonstrates that domain-specific statistics can yield meaningful practical speedups without increasing asymptotic complexity.

Abstract

In this work, we propose an enhancement to the Boyer-Moore-Horspool algorithm tailored for natural language text. The approach involves preprocessing the search pattern to identify its statistically least frequent character, referred to as the "anchor." During the search, verification is first performed at this high-entropy position, allowing the algorithm to quickly discard non-matching windows. This fail-fast strategy reduces unnecessary comparisons, improving overall efficiency. Our implementation shows that incorporating basic linguistic statistics into classical pattern-matching techniques can boost performance without increasing complexity to the shift heuristics.
Paper Structure (32 sections, 8 equations, 4 figures, 2 tables)

This paper contains 32 sections, 8 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Proposed Algorithm
  4. Preprocessing Phase
  5. Character Frequency Table
  6. Anchor Selection
  7. Shift Table Construction
  8. Searching Phase
  9. Anchor-First Verification
  10. Shift Calculation
  11. Complete Algorithm
  12. Example Execution
  13. Complexity Analysis
  14. Time Complexity
  15. Preprocessing Phase
  16. ...and 17 more sections

Figures (4)

  • Figure 1: Comparison of character comparisons across all four algorithms. Note the logarithmic scale highlighting the efficiency of shift-based approaches.
  • Figure 2: Direct comparison between FBAS and BMH algorithms. FBAS consistently requires fewer comparisons across all tested patterns.
  • Figure 3: Percentage improvement of FBAS over BMH for each pattern. Patterns with rare characters (z, b, g) show the highest gains.
  • Figure 4: Speedup factor relative to naive search. Both BMH and FBAS demonstrate order-of-magnitude improvements, with FBAS maintaining a consistent edge.