Dequantified Diffusion-Schr{ö}dinger Bridge for Density Ratio Estimation

TL;DR

This work tackles density ratio estimation under challenging distributional shifts by jointly addressing density-chasm and support-chasm. It introduces the $D^3RE$ framework, combining the dequantified diffusion bridge interpolant (DDBI) and the dequantified Schrödinger bridge interpolant (DSBI) to provide robust, stable, and efficient DRE. The authors prove theoretical properties including bounded time-scores via Gaussian dequantization and a Schrödinger-bridge formulation through entropic optimal transport rearrangement, and demonstrate improved density-estimation and mutual-information estimation across synthetic and MNIST-based experiments. Overall, $D^3RE$ offers a principled, scalable approach to high-dimensional density-ratio estimation with practical impact on domain adaptation, information estimation, and generative modeling tasks.

Abstract

Density ratio estimation is fundamental to tasks involving $f$-divergences, yet existing methods often fail under significantly different distributions or inadequately overlapping supports -- the density-chasm and the support-chasm problems. Additionally, prior approaches yield divergent time scores near boundaries, leading to instability. We design $\textbf{D}^3\textbf{RE}$, a unified framework for \textbf{robust}, \textbf{stable} and \textbf{efficient} density ratio estimation. We propose the dequantified diffusion bridge interpolant (DDBI), which expands support coverage and stabilizes time scores via diffusion bridges and Gaussian dequantization. Building on DDBI, the proposed dequantified Schr{ö}dinger bridge interpolant (DSBI) incorporates optimal transport to solve the Schr{ö}dinger bridge problem, enhancing accuracy and efficiency. Our method offers uniform approximation and bounded time scores in theory, and outperforms baselines empirically in mutual information and density estimation tasks.

Figures (24)

• Figure 1: Trajectory sets comparison among DI, DBI, DDBI and DSBI. Our methods yield broader trajectory sets, with intermediate distributions exhibiting wider support than those of DI. The entropically regularized transport losses for subfigures (a-d) are $44.17$, $31.14$, $31.15$, and $8.26$, respectively (See \ref{['eq:entropic-OT-problem']} for details.). Lower loss indicates increased path diversity.
• Figure 2: Density estimation results on eight 2-D synthetic datasets for different methods. $\text{D}^3\text{RE}$ effectively estimates the density for both multi-modal and discontinuous distributions.
• Figure 3: Evolution of estimated MI across epochs with varying methods and dimensions $d = \{40, 80, 120\}$. $\text{D}^3\text{RE}$ outperforms the baseline in both speed and precision. DRE-$\infty$choi2022density is regarded as the 'baseline' in this experiment.
• Figure 4: Ablation study on the effect of $\gamma^2$ for density estimation on 2-D toy data. The first row displays the results for the ground truth data. Each subsequent row, from top to bottom, corresponds to $\gamma^2$ values of 0.001, 0.01, 0.1, 0.5, and 1.0, respectively.
• Figure 5: Evolution of estimated MI across epochs with varying $\gamma^2=\{0.001,0.01,0.1,1.0\}$ and dimensions $d = \{40, 80, 120\}$. For all dimensions ($d = \{40, 80, 120\}$), smaller $\gamma^2$ values ($\leq 0.01$) lead to faster convergence.

Theorems & Definitions (17)

• Definition 3.1: Support-chasm Problem
• Theorem 3.2: Support Set Expansion
• Corollary 3.3: Trajectory Set Expansion
• Theorem 3.4
• Proposition 3.5
• Proposition 3.6
• Theorem 3.7
• Corollary 3.8
• Definition 3.9: Dequantified Diffusion bridge Density Ratio Estimation, D$^3$RE
• proof