Can Continuous-Time Diffusion Models Generate and Solve Globally Constrained Discrete Problems? A Study on Sudoku

TL;DR

This paper examines whether standard continuous-time diffusion models can represent distributions over highly constrained discrete structures, using completed Sudoku grids as a stress test and modeling along a Gaussian probability path $p_t(x|z)=\mathcal{N}(x;\alpha_t z,\beta_t^2 I)$. It systematically compares flow matching and score-based approaches, along with DDPM-style discretizations, across deterministic (ODE) and stochastic (SDE) samplers, and across unconditional generation and guided constraint solving with hard clues. Key findings show that unconditional generation is dominated by stochastic, score-based methods, with DDPM-style samplers delivering the highest validity among diffusion approaches; under guidance, linear probability paths with $\beta(t)$-scaled noise and score SDEs become competitive, reducing the gap with DDPM-based solvers. These results demonstrate that classic diffusion/flow formulations can assign non-zero probability mass to globally constrained discrete configurations and can be repurposed for constraint satisfaction via stochastic search, offering implications for using continuous-time diffusion in discrete, structured reasoning tasks. The study also highlights limitations in efficiency and generalizability, motivating future work on adaptive noise control and hybrid approaches that integrate diffusion with symbolic reasoning.

Abstract

Can standard continuous-time generative models represent distributions whose support is an extremely sparse, globally constrained discrete set? We study this question using completed Sudoku grids as a controlled testbed, treating them as a subset of a continuous relaxation space. We train flow-matching and score-based models along a Gaussian probability path and compare deterministic (ODE) sampling, stochastic (SDE) sampling, and DDPM-style discretizations derived from the same continuous-time training. Unconditionally, stochastic sampling substantially outperforms deterministic flows; score-based samplers are the most reliable among continuous-time methods, and DDPM-style ancestral sampling achieves the highest validity overall. We further show that the same models can be repurposed for guided generation: by repeatedly sampling completions under clamped clues and stopping when constraints are satisfied, the model acts as a probabilistic Sudoku solver. Although far less sample-efficient than classical solvers and discrete-geometry-aware diffusion methods, these experiments demonstrate that classic diffusion/flow formulations can assign non-zero probability mass to globally constrained combinatorial structures and can be used for constraint satisfaction via stochastic search.

Figures (6)

• Figure 1: Comparison of the solved rate metric as a function of the noise level $\sigma$ for SDE with only a trained score model(flow derived analytically from it), and SDE with both score and flow trained; both either with diffusion coefficient $\sigma$ or $\beta(t)$. Markers indicate the maximum value for each curve, with the corresponding $\sigma$ annotated.
• Figure 2: Entropy under linear schedule. For each plot we chose $\sigma$ which had the best success rate. Top row shows SDE variants with canonical Brownian noise; bottom row shows the corresponding variants with $\beta_t$ instead of $\sigma$ in the simulator. Columns compare having only score model trained or only flow trained, or combining both flow and score models trained separately.
• Figure 3: Entropy under cosine schedule for the Gaussian probability path. Right column compares SDE score with diffusion coefficient equal to either $\sigma$ or $\beta(t)$. Middle column compares DDPM samplers (DDIM vs DDPM). ODE results (with a velocity field derived from the trained score) are shown adjacent to DDIM for reference.
• Figure 4: Efficiency comparison across noise schedules and diffusion parameterizations for Easy, Medium, and Hard Sudoku instances. Bars show mean efficiency over runs with one standard deviation. Linear schedules consistently outperform cosine schedules, motivating their use in subsequent experiments.
• Figure 5: Empirical CDF of solving time on hard Sudoku instances under two runtime measures.