FLASH Viterbi: Fast and Adaptive Viterbi Decoding for Modern Data Systems

TL;DR

FLASH Viterbi introduces a fast, lightweight, adaptive Viterbi decoding framework optimized for edge devices. By replacing recursion with a non-recursive divide-and-conquer, applying pruning to remove inter-subtask dependencies, and optionally integrating dynamic beam search (FLASH-BS Viterbi), the approach achieves strong time and space efficiency. The hardware-friendly design, including an FPGA accelerator and double-buffered memory, demonstrates real-time performance with significantly reduced resource usage. The work provides extensive empirical results showing superior speedups and memory reductions over baselines, confirming practical impact for edge-stream inference in HMM-based pipelines.

Abstract

The Viterbi algorithm is a key operator for structured sequence inference in modern data systems, with applications in trajectory analysis, online recommendation, and speech recognition. As these workloads increasingly migrate to resource-constrained edge platforms, standard Viterbi decoding remains memory-intensive and computationally inflexible. Existing methods typically trade decoding time for space efficiency, but often incur significant runtime overhead and lack adaptability to various system constraints. This paper presents FLASH Viterbi, a Fast, Lightweight, Adaptive, and Hardware-Friendly Viterbi decoding operator that enhances adaptability and resource efficiency. FLASH Viterbi combines a non-recursive divide-and-conquer strategy with pruning and parallelization techniques to enhance both time and memory efficiency, making it well-suited for resource-constrained data systems. To further decouple space complexity from the hidden state space size, we present FLASH-BS Viterbi, a dynamic beam search variant built on a memory-efficient data structure. Both proposed algorithms exhibit strong adaptivity to diverse deployment scenarios by dynamically tuning internal parameters. To ensure practical deployment on edge devices, we also develop FPGA-based hardware accelerators for both algorithms, demonstrating high throughput and low resource usage. Extensive experiments show that our algorithms consistently outperform existing baselines in both decoding time and memory efficiency, while preserving adaptability and hardware-friendly characteristics essential for modern data systems. All codes are publicly available at https://github.com/Dzh-16/FLASH-Viterbi.

Figures (11)

• Figure 1: Theoretical time-space complexity comparison of FLASH Viterbi, FLASH-BS Viterbi, and baseline algorithms. $K$ is the HMM state space size, $T$ is the sequence length, $B$ is the beam width, and $P$ is the parallelism degree.
• Figure 2: Trellis diagram of Viterbi decoding, showing the optimal hidden state sequence ($q_t^*$) for the observation sequence ($x_t$).
• Figure 3: Comparison of recursive, non-recursive, and parallel non-recursive divide-and-conquer Viterbi decoding. Each rectangle represents a decoding subtask. A subtask labeled $i: (a,b)\_t$ indicates the $i$-th subtask that decodes the segment $(a,b)$ and outputs the optimal path state at timestep $t$, i.e., $q^*_t$.
• Figure 4: FLASH Viterbi subtask decoding process.
• Figure 5: Efficient data structure for dynamic beam search.

Theorems & Definitions (7)

• Definition 1: Valid path set
• Definition 2: Optimal path
• Definition 3: Retained-pruning variant
• Definition 4: Complete-pruning variant
• Theorem 1
• Theorem 2
• Theorem 3