Adversarially-Robust Inference on Trees via Belief Propagation
Samuel B. Hopkins, Anqi Li
TL;DR
The paper analyzes posterior inference on depth-$t$, $d$-regular trees under adversarial leaf corruption. It shows a sharp distinction between worst-case and semirandom adversaries: a ρ-fraction adversary can render the root unidentifiable, while a semirandom adversary with ρ small enough allows accurate root inference when the signal-to-noise ratio $d\varepsilon^2$ exceeds a logarithmic factor above the KS threshold, with belief propagation achieving this robust recovery. The main technique is a contraction argument for belief propagation, carefully handling long-range adversarial dependencies by decomposing adversaries into local components and, in the large-ε regime, employing truncation and derivative bounds. The results also connect robustness to model misspecification, provide a two-stage algorithm for spread adversaries, and establish information-theoretic lower bounds detailing the limits of adversarial robustness. Overall, the work demonstrates information-theoretically optimal corruption tolerance up to a constant factor and identifies open questions about robustness exactly at or below the KS threshold, with implications for robust Bayesian inference on tree-structured models.
Abstract
We introduce and study the problem of posterior inference on tree-structured graphical models in the presence of a malicious adversary who can corrupt some observed nodes. In the well-studied broadcasting on trees model, corresponding to the ferromagnetic Ising model on a $d$-regular tree with zero external field, when a natural signal-to-noise ratio exceeds one (the celebrated Kesten-Stigum threshold), the posterior distribution of the root given the leaves is bounded away from $\mathrm{Ber}(1/2)$, and carries nontrivial information about the sign of the root. This posterior distribution can be computed exactly via dynamic programming, also known as belief propagation. We first confirm a folklore belief that a malicious adversary who can corrupt an inverse-polynomial fraction of the leaves of their choosing makes this inference impossible. Our main result is that accurate posterior inference about the root vertex given the leaves is possible when the adversary is constrained to make corruptions at a $ρ$-fraction of randomly-chosen leaf vertices, so long as the signal-to-noise ratio exceeds $O(\log d)$ and $ρ\leq c \varepsilon$ for some universal $c > 0$. Since inference becomes information-theoretically impossible when $ρ\gg \varepsilon$, this amounts to an information-theoretically optimal fraction of corruptions, up to a constant multiplicative factor. Furthermore, we show that the canonical belief propagation algorithm performs this inference.