Dynamic programming by polymorphic semiring algebraic shortcut fusion

TL;DR

This paper introduces a principled framework for deriving dynamic programming algorithms from formal problem specifications using semiring polymorphism, shortcut fusion, and constraint lifting. By representing DP objectives as semiring expressions and employing a fusion theorem, the authors convert exhaustive generate-evaluate procedures into efficient, provably correct recurrences. Constraint lifting extends the approach to complex combinatorial problems by lifting semirings over separable constraint algebras, enabling unified handling of segmentation, sequence alignment, and discrete event problems, among others, with quantified complexity trade-offs. The framework also provides practical tools such as the tupling trick to avoid backtracking and a method to derive new DP algorithms from existing ones, demonstrated across multiple applications and accompanied by public Python software. Overall, the work offers a rigorous, modular methodology for exact DP algorithm design with broad applicability in signal processing, bioinformatics, and reliability engineering, while outlining future directions in parallelization and more advanced speedups.

Abstract

Dynamic programming (DP) is an algorithmic design paradigm for the efficient, exact solution of otherwise intractable, combinatorial problems. However, DP algorithm design is often presented in an ad-hoc manner. It is sometimes difficult to justify algorithm correctness. To address this issue, this paper presents a rigorous algebraic formalism for systematically deriving DP algorithms, based on semiring polymorphism. We start with a specification, construct an algorithm to compute the required solution which is self-evidently correct because it exhaustively generates and evaluates all possible solutions meeting the specification. We then derive, through the use of shortcut fusion, an implementation of this algorithm which is both efficient and correct. We also demonstrate how, with the use of semiring lifting, the specification can be augmented with combinatorial constraints, showing how these constraints can be fused with the algorithm. We furthermore demonstrate how existing DP algorithms for a given combinatorial problem can be abstracted from their original context and re-purposed. This approach can be applied to the full scope of combinatorial problems expressible in terms of semirings. This includes, for example: optimal probability and Viterbi decoding, probabilistic marginalization, logical inference, fuzzy sets, differentiable softmax, relational and provenance queries. The approach, building on ideas from the existing literature on constructive algorithmics, exploits generic properties of polymorphic functions, tupling and formal sums and algebraic simplifications arising from constraint algebras. We demonstrate the effectiveness of this formalism for some example applications arising in signal processing, bioinformatics and reliability engineering. Python software implementing these algorithms can be downloaded from: http://www.maxlittle.net/software/dppolyalg.zip.

Figures (3)

• Figure 1: DP segmentation algorithms derived using our novel algebraic framework for solving constrained, 1D segmented, least-squares linear regression, applied to synthetic, piecewise linear time series with i.i.d. Gaussian noise, standard deviation $\sigma$. Input data $y_{n}$ (grey dots), underlying piecewise constant signal (grey line), segmentation result (red line). (a) Unconstrained segmentation with regularization $\lambda=15$, noise $\sigma=15$, (b) with fixed number segments $L=3$, noise $\sigma=30$, (c) with minimum segment length $M=70$, noise $\sigma=60$, and (d) for comparison, L1 trend filtering with regularization $\lambda=10^{3}$.
• Figure 2: DP segmentation algorithms derived using our novel algebraic framework for solving constrained, 1D segmented, least-squares linear regression, applied to a sample of logarithmically-transformed S&P500 financial index daily values. Input data $y_{n}$ (grey lines), segmentation result (red line). (a) Unconstrained segmentation with regularization $\lambda=1.78\times10^{-5}$ , (b) with fixed number segments $L=4$, (c) with minimum segment length $M=50$ days, and (d) for comparison, L1 trend filtering with regularization $\lambda=100$.
• Figure 3: Computational time (black line) required to solve the Needleman-WunschDP sequence alignment algorithm (left) without constraints and (right) with lifted constraint. The horizontal axis is the length of both sequences and also the size of the constraint algebra (e.g. $N=M=\left|\mathcal{M}\right|$). The vertical axis is on a quadratic (left) and cubic (right) scale such that exact $O\left(N^{2}\right)$ and $O\left(N^{3}\right)$ complexities correspond to a straight line (grey line). Python language implementation on a quad-core Intel Core i7 3.2GHz, 16Gb DRAM.

Theorems & Definitions (3)

• Theorem 1
• Corollary
• Theorem 2