The Marked Edge Walk: A Novel MCMC Algorithm for Sampling of Graph Partitions

TL;DR

The paper addresses the challenge of sampling redistricting partitions from distributions not tied to spanning-tree counts. It introduces the marked edge walk (MEW), an MCMC algorithm that operates on the lifted space $\mathcal{X}=\{(T,M): T\text{ is a spanning tree}, M\subseteq T\}$ by combining a cycle-basis move on $T$ with a marked-edge move on $M$, and uses Metropolis-Hastings with tractable forward/backward probabilities. MEW enables targeting arbitrary energy-based distributions $p(x)\propto\exp(J(\xi(x)))/\tau(\xi(x))$ over partitions, where $\tau(\xi)$ accounts for degeneracy from spanning trees, thereby removing the overwhelming bias of spanning-tree counts and allowing assessment of compactness, competitiveness, and multivariate criteria. Across Cheshire County, New Hampshire, and Texas, MEW demonstrates fast mixing, convergence to target distributions, and scalability to large graphs, while highlighting a bias toward slightly less compact plans under certain targets and noting computational costs from degeneracy calculations. This work broadens ensemble generation capabilities in redistricting and supports prospective policy analysis and optimization by enabling explicit targeting of diverse partition criteria.

Abstract

Novel Markov Chain Monte Carlo (MCMC) methods have enabled the generation of large ensembles of redistricting plans through graph partitioning. However, existing algorithms such as Reversible Recombination (RevReCom) and Metropolized Forest Recombination (MFR) are constrained to sampling from distributions related to spanning trees. We introduce the marked edge walk (MEW), a novel MCMC algorithm for sampling from the space of graph partitions under a tunable distribution. The walk operates on the space of spanning trees with marked edges, allowing for calculable transition probabilities for use in the Metropolis-Hastings algorithm. Empirical results on real-world dual graphs show convergence under target distributions unrelated to spanning trees. For this reason, MEW represents an advancement in flexible ensemble generation.

Figures (9)

• Figure 1: Illustration of MEW's two-step procedure. (Top-left) Initial state with spanning tree (black edges) and marked edges (red) that create a balanced partition when removed. (Top-right) Cycle basis step: adding edge $e_+$ (green) creates cycle $C$ (blue), then removing edge $e_-$ (orange) yields new tree $T'$. (Bottom-left) Marked edge step: the right vertex of the marked edge (red) is chosen, and its right neighbor is chosen, yielding a new marked edge (pink). (Bottom-right) These two steps create a new partition, completing one step of the MEW procedure.
• Figure 1: Example $x, x'$ that breaks symmetry
• Figure 2: Comparison of a sampled Texas redistricting plan (left) and the enacted 2020 plan (right). The spanning tree products differ by approximately $10^{466}$, demonstrating the extreme and impractical nature of spanning tree-based probability weighting in real-world applications.
• Figure 3: Comparison of cut edges for enumerated, sampled, and unsampled partitions in Cheshire County’s dual graph. After running MEW for 1 million steps, 99% of the 34,225 balanced 2-partitions were sampled. Unsampled partitions (green) tend to have higher cut edges than sampled partitions (orange).
• Figure 4: (a) Comparison of MEW (blue) to a Recombination baseline (red) when sampling with a target of the spanning tree distribution. (b) Decay of the Average Pairwise KS Distance (blue) and the Average Kolmogorov-Smirnov (KS) Distance to the Spanning Tree Distribution (red). After many iterations, MEW successfully samples from the spanning tree distribution, as evidenced by its convergence to the baseline.