Reduction of interaction order in hard combinatorial optimization via conditionally independent degrees of freedom

TL;DR

This work introduces a real-space renormalization group framework that reduces third-order 3-SAT interactions to pairwise (QUBO) form while preserving the free energy, by adding extra spins and exploiting conditional independence. The key insight is that these extra spins can be traced out exactly, yielding an effective field on the original spins and enabling a Monte Carlo procedure (QUBO-Tr) that matches PUBO performance without sampling the auxiliary degrees of freedom. A parallel-update variant in the $\beta\to\infty$ limit (QUBO-Tr $\infty$) provides a transparent link to the RG spectrum and helps explain the observed landscape ruggedness, while QUBO-RG offers a practical, hardware-compatible pathway to implement higher-order problems with quadratic interactions. The approach promises improved scalability for Ising-machine solvers, with potential extensions to higher-order constraints and connections to belief propagation and Markov chain methods.

Abstract

Combinatorial optimization problems have a broad range of applications and map to physical systems with complex dynamics. Among them, the 3-SAT problem is prominent due to its NP-complete nature. In physics terms, its solution corresponds to finding the ground state of a disordered Ising spin Hamiltonian with third-order, or tensor, interactions. The large growth of the number of third-order interactions with number of variables poses technical difficulties for the physical implementation of minimizers. Therefore, researchers have proposed quadratization techniques which reduce the order of the system, however, at the cost of including additional degrees of freedom. Their inclusion induces a drastic slow down in the minimization, which makes such procedures technically infeasible for large problems. In this work, we take a physics approach by employing the renormalization group to create a pairwise interacting system from the original third-order system while preserving the free energy. Our procedure utilizes additional degrees of freedom that exhibit an independent dynamics provided the original degrees of freedom are fixed. A step-wise trace of the extra variables while running the minimization is therefore theoretically manageable, yielding a state-dependent effective interaction. We use the effective interaction to reconstruct the original third-order energy spectrum, as this yields equal scaling of computations-to-ground-state compared to the original tensor formulation. Here, the original degrees of freedom interact with a subsystem that appears to be in a superposition of an exponentially large number of states. In the zero-temperature limit, the superposition concentrates on one state. Our spectrum-engineering techniques reveal new routes toward the ground state of disordered Ising systems, through Markov chains, and allow for efficient technological implementations.

Figures (13)

• Figure 1: System OverviewI) Original system with third-order interactions (PUBO) and $N$ base spins (blue). II) The inverse RG transform, Eqs.$~\ref{['eq:RG']}$ and \ref{['Hq']}, maps PUBO to a system with only pairwise interactions QUBO-RG, but with $N_{e}$ extra spins (red) in addition. III) Performing the trace over the extra spins' configurations is done in QUBO-Tr, Eq.$~\ref{['eq:dissect']}$, yielding the original PUBO local field, where the extra spins play the role of an effective field acting on each base spin. IV) We link the $\beta\rightarrow\infty$ limit of the traced system with the long runtime ($t\rightarrow\infty$) limit of QUBO-RG, which is a novel way to interpret the concept of "ruggedness" in systems with a fixed temperature, in Fig.$~\ref{['fig:fraction_red']}$. (a) Each extra spin $s_{e}$ has a pairwise interaction with only three base spins $s_{i}$, $s_{j}$, $s_{k}$, stemming from the original third order interaction $L_{ijk}s_{i}s_{j}s_{k}$. (b) Extra spins are traced out, which amounts to taking into account all of their $2^{N_{e}}$ configurations at once, in a superposition, and giving rise to an additional effective field Eq.$~\ref{['eq:dissect']}$ acting on spin $s_{i}$ to be updated. (c) In the limit of $\beta\to\infty$, the superposition of extra spins becomes apparent: Depending on the base spin to be updated (indicated by red rectangle, top vs bottom row describe two different updates), the extra spins (red arrows) appear to reside in a different state, described by their local field $\bar{h}_{e}\backslash_{i}$ in Eq.$~\ref{['eq:parallel']}$. (d) This local field $\bar{h}_{e}\backslash_{i}$ corresponds to a cavity field imposed on $s_{i}$, which is independent of the value of $s_{i}$ and is interpreted as originating from a fictitious extra spin $s_{e}^{*}$, see Eq.$~\ref{['eq:fict']}$.
• Figure 2: Density plot of local fields of PUBO Eq.$~\ref{['eq:H_normal']}$ and QUBO-Tr Eq.$~\ref{['eq:dissect']}$. Run on an $N=100$ instance for $\tau_{s}=10^{5}$ steps and temperature with $\beta=5$. At each step the two fields are separately calculated for the same spin selected for update. See App.$~\ref{['sec:Comparison_QUBO_PUBO']}$ for the comparison of QUBO-RG and PUBO fields.
• Figure 3: Performance comparison between the classical quadratization technique by Rosenberg 10558658 and PUBO Eq.$~\ref{['eq:H_normal']}$, and our novel approaches: QUBO-RG from Eq.$~\ref{['Hq']}$, QUBO-Tr($\infty$) from Eq.$~\ref{['eq:parallel']}$, and QUBO-Tr from Eq.$~\ref{['eq:dissect']}$; We show values up to $10^{8}$ due to corresponding runtimes being exceedingly large. The parallel tempering heuristic PhysRevLett.57.2607 is used to accelerate the spin updates needed to achieve a solution. It consists of running our fixed-temperature solvers at many temperatures in parallel (each temperature is a replica) and properly exchanging temperatures after each run. Spin flips calculated in parallel were not counted in the UTS. For each solver, 12 replicas are used, hence there are 12 update steps performed in parallel, one for each parallel tempering replica. The temperature schedule is $\beta=3n,\quad\text{with}n\in\{1,2,...,12\}$, and a single swap of temperatures is performed. The two runs at a given temperature, one before and one after the swap, were included in UTS. See App.$~\ref{['sec:pt_schedule']}$ for a schedule justification and details on the heuristic. Problem instances are random CSPs, retrieved from the data set SATLIB ref.hoos_satlib number uf$N$-[901,1000]. $\mathrm{UTS}_{99}=\tau_{s}\log(0.01)/\log(1-p_{s})$ determined from the success probability $p_{s}$ which is estimated by the ratio of successful solutions among 100 attempts, each started from a random initial condition.
• Figure 4: Density plot of local fields of PUBO Eq.$~\ref{['eq:H_normal']}$ and QUBO-Tr($\infty$) Eq.$~\ref{['eq:fict']}$. Run on an $N=100$ instance for $\tau_{s}=10^{5}$ steps, where at each step the two fields are separately calculated at the spin selected for update, with $\beta=5$ (left) and $\beta=100$ (right), respectively.
• Figure 5: Comparison of QUBO-RG and QUBO-Tr($\infty$). Both algorithms are applied to $N=100$ base spin problems for a range of temperatures in parallel, averaged over 100 instances and 100 random initial conditions each. The number of spin updates for each solver $T_{sim}^{\mathrm{QUBO-RG}}$and $T_{sim}^{\mathrm{QUBO-Tr}(\infty)}$ is calculated as $T_{sim}^{\mathrm{QUBO-RG}}=T_{WC}\times N_{\mathrm{QUBO-RG}}$ and $T_{sim}^{\mathrm{QUBO-Tr}(\infty)}=T_{WC}\times N_{\mathrm{\mathrm{QUBO-Tr}(\infty)}},$ where $T_{WC}$ is the wall-clock steps and $N_{QUBO-RG},N_{QUBO-Tr(\infty)}$ are the number of spins that each solver runs on respectively. Differences for long runtime between the curves are due to different local fields prior to the final time step. The inset explains the similarity of the curves at the plotted scale, where a QUBO-RG run is applied at one problem with $N=100$ and $\beta=10$. During such a run, we keep track of the quantity $\mathcal{B}^{\mathrm{QUBO-RG}}(\bar{h}_{e}\backslash_{i})$, which counts the fraction of update steps of spin $i$ during which all its connected extra spins have a configuration predicted by $\mathrm{sgn}(\bar{h}_{e}\backslash_{i})$; such extra spins thus follow the $\beta\to\infty$ dynamics \ref{['eq:parallel']}. The fraction $\mathcal{B}^{\mathrm{QUBO-RG}}(\overline{h_{e}}^{Q})$ counts the fraction of update steps of spin $i$ during which all its connected extra spins have a configuration predicted by $\mathrm{sgn}(\bar{h}_{e}^{Q})$. As an example, if 15 out of the 16 extra spins connected to a base spin are correctly aligned, this does not increase the count, see App.$~\ref{['sec:qubo_dynamics']}$.