On the Worst-Case Complexity of Gibbs Decoding for Reed--Muller Codes
Xuzhe Xia, Nicholas Kwan, Lele Wang
TL;DR
This work analyzes the worst-case efficiency of Gibbs decoding for Reed–Muller codes on the binary symmetric channel by sampling from the posterior. It proves there exist channel outputs for which the Gibbs sampler mixes extremely slowly, with a lower bound $T_{\mathrm{mix}}=\Omega\left(\exp(\sqrt{n}\cdot \exp(-\sqrt{\log n}))\right)$, implying super-polynomial time to converge. The proof uses a bottleneck/conductance argument together with an explicit construction of a typical-looking output $\mathbf{y}$ that creates a bottleneck near the all-zero message, showing a fundamental obstacle to polynomial-time RM decoding via Gibbs sampling in the worst case. The results suggest that while posterior sampling preserves capacity-achieving guarantees in principle, practical RM decoding may require alternative MCMC schemes (e.g., block updates or annealing) or different decoder designs; average-case behavior and applicability to other MCMC decoders remain open. Overall, the work highlights a notable complexity barrier for Bayesian decoding approaches on RM codes.
Abstract
Reed--Muller (RM) codes are known to achieve capacity on binary symmetric channels (BSC) under the Maximum a Posteriori (MAP) decoder. However, it remains an open problem to design a capacity achieving polynomial-time RM decoder. Due to a lemma by Liu, Cuff, and Verdú, it can be shown that decoding by sampling from the posterior distribution is also capacity-achieving for RM codes over BSC. The Gibbs decoder is one such Markov Chain Monte Carlo (MCMC) based method, which samples from the posterior distribution by flipping message bits according to the posterior, and can be modified to give other MCMC decoding methods. In this paper, we analyze the mixing time of the Gibbs decoder for RM codes. Our analysis reveals that the Gibbs decoder can exhibit slow mixing for certain carefully constructed sequences. This slow mixing implies that, in the worst-case scenario, the decoder requires super-polynomial time to converge to the desired posterior distribution.