What is the most optimal diffusion?
Vasili Baranau
TL;DR
The paper tackles the problem of defining the fastest possible diffusion by maximizing the differential entropy of a pdf at each time step under a fixed KL-divergence energy constraint, leading to a nonlocal integro-differential evolution distinct from classical Itô diffusion. The authors derive the explicit evolution equation $\frac{\partial f}{\partial t} = -\kappa f\left(\ln f - \int f\ln f \,d\vec{x}\right)$ with $\kappa$ determined by the KL constraint, and show that this diffusion is nonlocal and can outperform classical diffusion in entropy growth. They identify exact 1D solutions (normal, exponential) with variance growing as $\operatorname{Var}\sim e^{2A t}$, and demonstrate a truncated-normal solution on a segment, supported by numerical experiments on sigmoid-like functions. The results suggest a novel, potentially practical framework for diffusion-based modeling and invite further exploration of nonlocal dynamics and diffusion-inspired generative methods.
Abstract
What is the fastest possible "diffusion"? A trivial answer would be "a process that converts a Dirac delta-function into a uniform distribution infinitely fast". Below, we consider a more reasonable formulation: a process that maximizes differential entropy of a probability density function (pdf) $f(\vec{x}, t)$ at every time $t$, under certain restrictions. Specifically, we focus on a case when the rate of the Kullback-Leibler divergence $D_{\text{KL}}$ is fixed. If $Δ(\vec{x}, t, d{t}) = \frac{\partial f}{ \partial t} d{t}$ is the pdf change at a time step $d{t}$, we maximize the differential entropy $H[f + Δ]$ under the restriction $D_{\text{KL}}(f + Δ|| f) = A^2 d{t}^2$, $A = \text{const} > 0$. It leads to the following equation: $\frac{\partial f}{ \partial t} = - κf (\ln{f} - \int f \ln{f} d{\vec{x}})$, with $κ= \frac{A}{\sqrt{ \int f \ln^2{f} d{\vec{x}} - \left( \int f \ln{f} d{\vec{x}} \right)^2 } }$. Notably, this is a non-local equation, so the process is different from the Itô diffusion and a corresponding Fokker-Planck equation. We show that the normal and exponential distributions are solutions to this equation, on $(-\infty; \infty)$ and $[0; \infty)$, respectively, both with $\text{variance} \sim e^{2 A t}$, i.e. diffusion is highly anomalous. We numerically demonstrate for sigmoid-like functions on a segment that the entropy change rate $\frac{d H}{d t}$ produced by such an optimal "diffusion" is, as expected, higher than produced by the "classical" diffusion.