Strong Data Processing Inequalities and their Applications to Reliable Computation

TL;DR

This work addresses the problem of computing Boolean functions using unreliable components by establishing noise thresholds for reliable computation and connecting constructive circuit results with strong information-theoretic inequalities. It combines von Neumann’s results on 3-input gates with Strong Data-Processing Inequalities (SDPI) and Bayesian-network analyses to derive sharp contraction bounds on information flow, culminating in Evans–Schulman’s bound $δ^*(k) ≤ δ^*_{ES}(k) = \tfrac{1}{2} - \tfrac{1}{2\sqrt{k}}$ and its interpretation via percolation on Bayesian networks. The analysis demonstrates that for sufficiently small gate-noise $δ$, 3-input majority and minority circuits can reliably compute any Boolean function, and it shows how SDPI techniques yield general upper bounds for noisy circuit reliability, aligning with earlier formula-based results for odd fan-in. The findings illuminate the interplay between circuit structure, noise resilience, and information contraction, with implications for understanding reliable computation in noisy hardware and connections to brain-inspired information processing; several open questions remain for even fan-in and precise thresholds for specific gate families.

Abstract

In 1952, von Neumann gave a series of groundbreaking lectures that proved it was possible for circuits consisting of 3-input majority gates that have a sufficiently small independent probability $δ> 0$ of malfunctioning to reliably compute Boolean functions. In 1999, Evans and Schulman used a strong data-processing inequality (SDPI) to establish the tightest known necessary condition $δ< \frac{1}{2} - \frac{1}{2\sqrt{k}}$ for reliable computation when the circuit consists of components that have at most $k$ inputs. In 2017, Polyanskiy and Wu distilled Evans and Schulman's SDPI argument to establish a general result on the contraction of mutual information in Bayesian networks. In this essay, we will first introduce the problem of reliable computation from unreliable components and establish the existence of noise thresholds. We will then provide an exposition of von Neumann's result with 3-input majority gates and extend it to minority gates. We will then provide an introduction to SDPIs, which have many applications, including in statistical mechanics, portfolio theory, and lower bounds on statistical estimation under privacy constraints. We will then use the introduced material to provide an exposition of Polyanskiy and Wu's 2017 result on Bayesian networks, from which the 1999 result of Evans-Schulman follows.

Figures (8)

• Figure 1: Example circuit and corresponding DAG
• Figure 2: Scatter points mark the values at positive integers and odd positive integers for $\delta^*_{\text{ES}}$ and $\delta^*_f$ respectively.
• Figure 3: Construction of $C^*$ in the case $n=5$. This is the first level of the induction, where $C$ has one output and computes $f$.
• Figure 4: Construction of $D^*$ in the case $n=5$ and $D$ has 2 outputs ($m=2$).
• Figure 5: Plot of cubic polynomial $p_\delta (\eta)$ in the 3MIN gates proof for three different values of $\delta$.

Theorems & Definitions (60)

• Definition 2.1: Computation by Boolean Circuits
• Theorem 2.2: Shannon, 1938 S38. Proof omitted.
• Definition 2.3: Noisy Circuits
• Remark
• Definition 2.4: Reliable Computation
• Remark
• Lemma 2.5
• proof
• Definition 2.6: Fan-in and Fan-out
• Theorem 2.7: Evans and Schulman, 1999 ES99.