Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coarse-Grained Kullback--Leibler Control of Diffusion-Based Generative AI

Tatsuaki Tsuruyama

TL;DR

The paper addresses the lack of a theory for how coarse-grained image statistics are preserved during diffusion-based generative sampling. It introduces a leak-tolerant Kullback–Leibler potential $V_\delta$ and couples it with a two-step $V_\delta$-projected reverse diffusion that first performs a standard update and then projects onto a coarse-grained constraint set, preserving block-level statistics. It proves that the Lyapunov property of $V$ extends to time-inhomogeneous block-preserving kernels and that $V_\delta$ remains approximately monotone under small leakage, with a closed-form projection implemented via a scaling-and-clipping of block masses. Numerical experiments on a block-constant image toy model show that the projection keeps block-mass error within the prescribed tolerance $\delta$ while achieving pixel-level quality comparable to unprojected dynamics, suggesting a principled design for diffusion samplers with explicit coarse-grained control.

Abstract

Diffusion models and score-based generative models provide a powerful framework for synthesizing high-quality images from noise. However, there is still no satisfactory theory that describes how coarse-grained quantities, such as blockwise intensity or class proportions after partitioning an image into spatial blocks, are preserved and evolve along the reverse diffusion dynamics. In previous work, the author introduced an information-theoretic Lyapunov function V for non-ergodic Markov processes on a state space partitioned into blocks, defined as the minimal Kullback-Leibler divergence to the set of stationary distributions reachable from a given initial condition, and showed that a leak-tolerant potential V-delta with a prescribed tolerance for block masses admits a closed-form expression as a scaling-and-clipping operation on block masses. In this paper, I transplant this framework to the reverse diffusion process in generative models and propose a reverse diffusion scheme that is projected by the potential V-delta (referred to as the V-delta projected reverse diffusion). I extend the monotonicity of V to time-inhomogeneous block-preserving Markov kernels and show that, under small leakage and the V-delta projection, V-delta acts as an approximate Lyapunov function. Furthermore, using a toy model consisting of block-constant images and a simplified reverse kernel, I numerically demonstrate that the proposed method keeps the block-mass error and the leak-tolerant potential within the prescribed tolerance, while achieving pixel-wise accuracy and visual quality comparable to the non-projected dynamics. This study reinterprets generative sampling as a decrease of an information potential from noise to data, and provides a design principle for reverse diffusion processes with explicit control of coarse-grained quantities.

Coarse-Grained Kullback--Leibler Control of Diffusion-Based Generative AI

TL;DR

The paper addresses the lack of a theory for how coarse-grained image statistics are preserved during diffusion-based generative sampling. It introduces a leak-tolerant Kullback–Leibler potential and couples it with a two-step -projected reverse diffusion that first performs a standard update and then projects onto a coarse-grained constraint set, preserving block-level statistics. It proves that the Lyapunov property of extends to time-inhomogeneous block-preserving kernels and that remains approximately monotone under small leakage, with a closed-form projection implemented via a scaling-and-clipping of block masses. Numerical experiments on a block-constant image toy model show that the projection keeps block-mass error within the prescribed tolerance while achieving pixel-level quality comparable to unprojected dynamics, suggesting a principled design for diffusion samplers with explicit coarse-grained control.

Abstract

Diffusion models and score-based generative models provide a powerful framework for synthesizing high-quality images from noise. However, there is still no satisfactory theory that describes how coarse-grained quantities, such as blockwise intensity or class proportions after partitioning an image into spatial blocks, are preserved and evolve along the reverse diffusion dynamics. In previous work, the author introduced an information-theoretic Lyapunov function V for non-ergodic Markov processes on a state space partitioned into blocks, defined as the minimal Kullback-Leibler divergence to the set of stationary distributions reachable from a given initial condition, and showed that a leak-tolerant potential V-delta with a prescribed tolerance for block masses admits a closed-form expression as a scaling-and-clipping operation on block masses. In this paper, I transplant this framework to the reverse diffusion process in generative models and propose a reverse diffusion scheme that is projected by the potential V-delta (referred to as the V-delta projected reverse diffusion). I extend the monotonicity of V to time-inhomogeneous block-preserving Markov kernels and show that, under small leakage and the V-delta projection, V-delta acts as an approximate Lyapunov function. Furthermore, using a toy model consisting of block-constant images and a simplified reverse kernel, I numerically demonstrate that the proposed method keeps the block-mass error and the leak-tolerant potential within the prescribed tolerance, while achieving pixel-wise accuracy and visual quality comparable to the non-projected dynamics. This study reinterprets generative sampling as a decrease of an information potential from noise to data, and provides a design principle for reverse diffusion processes with explicit control of coarse-grained quantities.
Paper Structure (22 sections, 3 theorems, 48 equations, 2 figures)

This paper contains 22 sections, 3 theorems, 48 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Results
  3. Definition of $V_\delta$-projected reverse diffusion
  4. Closed form of the Vdelta projection and implementation properties
  5. Lyapunov properties of Vdelta-projected reverse diffusion
  6. Interpretation as a splitting method
  7. Numerical experiments: reproducing image formation in a block model
  8. Block-constant image model
  9. Forward replication dynamics and limiting distribution
  10. Reverse kernel and baseline sampler
  11. Comparison with V-delta-projected reverse diffusion
  12. Image formation from random initial conditions
  13. Model quantities
  14. Block partition and block masses
  15. V(p): potential for within-block non-uniformity
  16. ...and 7 more sections

Key Result

Proposition 1

With the above notation, the equation has a unique solution $\tau^\star > 0$, and the block masses of $\Pi_\delta(p)$ are given by Moreover, if the within-block conditional distribution is kept unchanged, i.e., then the resulting distribution $\pi^\star$ coincides with $\Pi_\delta(p)$.

Figures (2)

  • Figure 1: Example of reverse diffusion from a block-constant image. Left: original block-constant image $q_{\mathrm{data}}$. Middle: initial state $p_0$ obtained by blockwise Gaussian blurring. Right: snapshots at representative steps $n$ for the non-projected reverse diffusion and the $V_\delta$-projected reverse diffusion.
  • Figure 2: Time evolution of information-theoretic indicators during reverse diffusion. Top: $V(p_n)$. Middle: leak-tolerant potential $V_\delta(p_n)$. Bottom: block-mass error $E_{\mathrm{block}}(n) = \|w(p_n) - w^{\mathrm{ref}}\|_1$. Solid lines correspond to the non-projected reverse diffusion; dashed lines correspond to the $V_\delta$-projected reverse diffusion. In the projected case, both $E_{\mathrm{block}}(n)$ and $V_\delta(p_n)$ remain within the tolerance controlled by $\delta$, yielding a stable generative process from the viewpoint of coarse-grained quantities.

Theorems & Definitions (3)

  • Proposition 1: Closed form of the $V_\delta$-projection at block-mass level
  • Theorem 1: Monotonicity for time-inhomogeneous block-preserving dynamics
  • Theorem 2: Approximate monotonicity of $V_\delta$-projected reverse diffusion