On k-Mer-Based and Maximum Likelihood Estimation Algorithms for Trace Reconstruction

TL;DR

This work investigates two algorithmic paradigms for the worst-case trace reconstruction problem: $k$-mer based methods and the Maximum Likelihood Estimator (MLE). It proves a tight lower bound for $k$-mer based algorithms, showing that even with optimal $k$-mer density statistics one cannot beat $\exp(\Omega(n^{1/5}\sqrt{\log n}))$ traces, and that the Chase upper bound is essentially tight within polylogarithmic factors. Conversely, it shows that the MLE, although exponential-time, is optimal in sample complexity up to a linear factor in $n$ for worst-case scenarios and matches average-case optimality without that blow-up. The technical backbone blends complex-analytic polynomial methods (via generating polynomials $P_{w,\mathbf{x}}$, Chebyshev expansions, Bernstein ellipses, and Hadamard three-circles) with a counting argument to bound distinguishability between strings, and a model-estimation perspective to bound the MLE’s performance. Overall, the paper delineates the limits of two natural strategies for trace reconstruction and highlights the need for new techniques to surpass the current upper bound on worst-case sample complexity, while confirming the near-optimality of MLE in broad settings and its fundamental role as a universal tool in trace reconstruction.

Abstract

The goal of the trace reconstruction problem is to recover a string $x\in\{0,1\}^n$ given many independent {\em traces} of $x$, where a trace is a subsequence obtained from deleting bits of $x$ independently with some given probability $p\in [0,1).$ A recent result of Chase (STOC 2021) shows how $x$ can be determined (in exponential time) from $\exp(\widetilde{O}(n^{1/5}))$ traces. This is the state-of-the-art result on the sample complexity of trace reconstruction. In this paper we consider two kinds of algorithms for the trace reconstruction problem. Our first, and technically more involved, result shows that any $k$-mer-based algorithm for trace reconstruction must use $\exp(Ω(n^{1/5}))$ traces, under the assumption that the estimator requires $poly(2^k, 1/\varepsilon)$ traces, thus establishing the optimality of this number of traces. The analysis of this result also shows that the analysis technique used by Chase (STOC 2021) is essentially tight, and hence new techniques are needed in order to improve the worst-case upper bound. Our second, simple, result considers the performance of the Maximum Likelihood Estimator (MLE), which specifically picks the source string that has the maximum likelihood to generate the samples (traces). We show that the MLE algorithm uses a nearly optimal number of traces, \ie, up to a factor of $n$ in the number of samples needed for an optimal algorithm, and show that this factor of $n$ loss may be necessary under general ``model estimation'' settings.

Theorems & Definitions (37)

• Definition 1: mazooji2022substring
• Definition 2
• Definition 3: Maximum Likelihood Estimation
• Theorem 1: Implied by chase2021separating
• Theorem 2
• Remark 1
• Theorem 3
• Corollary 1.1
• Theorem 4
• Remark 2