Multimodal Sampling via Approximate Symmetries

TL;DR

This work addresses sampling from multimodal distributions that lack exact symmetries by exploiting approximate group symmetries. The authors construct a nearby symmetric reference distribution via orbit averaging $E_R(s)=\frac{1}{|G|}\sum_{g\in G} E(gs)$, then apply continuation methods (AIS or TT) to travel from $E_R$ to the target $E$, reducing the distance along the continuation path and improving efficiency. The approach is demonstrated on Ising-model examples, where the symmetric reference enables faster mixing and yields meaningful mode probabilities with manageable weight variance. Overall, the framework extends symmetry-based acceleration to near-symmetric problems and broadens the applicability of multilevel sampling techniques to challenging multimodal distributions.

Abstract

Sampling from multimodal distributions is a challenging task in scientific computing. When a distribution has an exact symmetry between the modes, direct jumps among them can accelerate the samplings significantly. However, the distributions from most applications do not have exact symmetries. This paper considers the distributions with approximate symmetries. We first construct an exactly symmetric reference distribution from the target one by averaging over the group orbit associated with the approximate symmetry. Next, we can apply the multilevel Monte Carlo methods by constructing a continuation path between the reference and target distributions. We discuss how to implement these steps with annealed importance sampling and tempered transitions. Compared with traditional multilevel methods, the proposed approach can be more effective since the reference and target distributions are much closer. Numerical results of the Ising models are presented to illustrate the efficiency of the proposed method.

Figures (9)

• Figure 1: The main steps of the algorithms. Left: $G$ is the group of symmetries. $G$ is an approximate symmetry for the distribution with log density $E(s)$. Through averaging over the group orbit, we construct a reference log density $E_R(s)$, which is symmetric under $G$. Middle: annealed importance sampling (AIS), where $g\in G$ allows efficient sampling from the reference $E_R(s)$. Right: tempered transition (TT), where $g\in G$ allows for global jumps within a TT move.
• Figure 2: The triangle stands for the probability simplex defined over the state space. Left: the usual continuation method. The usual reference distribution with $\beta=0$ has a lot of symmetries and is easy to sample. However, it is often far from the target distribution $E(s)$, so the continuation path is long. Right: the proposed method. Based on the approximate symmetry $G$ of the $E(s)$, we identify a nearby reference distribution $E_R(s)$ with $G$ as the exact symmetry. The orange dots are the distributions that satisfy the exact symmetry in the group $G$ ($\beta=0$ is merely one of them). The continuation path from $E(s)$ to $E_R(s)$ is much shorter than the one from $E(s)$ to the $\beta=0$ distribution.
• Figure 3: Two dominant macroscopic profiles of the Ising model in Example 1. White stands for $+1$ while black for $-1$. The two dominant macroscopic profiles are a $-1$ cluster linking the left/right sides and a $+1$ cluster linking the top/bottom sides.
• Figure 4: Example 1. (a): the forcing term $f$ of $E(s)$. (b): the forcing term $\frac{1}{2} (f-Pf)$ of $E_R(s)$.
• Figure 5: The logarithms of the normalized weights \ref{['eq:wn']} of each sample over all levels.

Theorems & Definitions (6)

• Definition 1
• Definition 3
• Claim 4
• proof
• Claim 5
• proof