Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sliced Wasserstein with Random-Path Projecting Directions

Khai Nguyen, Shujian Zhang, Tam Le, Nhat Ho

TL;DR

The paper tackles the inefficiency of selecting slicing directions in Sliced Wasserstein distances by introducing Random-Path Projecting Direction (RPD) and Random-Path Slicing Distribution (RPSD), enabling optimization-free sampling. It defines two SW variants, Random-Path Projection Sliced Wasserstein (RPSW) and Importance Weighted RPSW (IWRPSW), and establishes their theoretical properties, including topological and statistical aspects and favorable computational complexity. The authors demonstrate that RPSW and IWRPSW outperform traditional SW variants in gradient-flow tasks and in training denoising diffusion models, achieving faster convergence and improved generation quality with reduced compute. This approach offers a scalable, discriminative, and practical alternative to optimization-based slicing schemes, with potential extensions to manifolds and a wide range of downstream applications in generative modeling and beyond.

Abstract

Slicing distribution selection has been used as an effective technique to improve the performance of parameter estimators based on minimizing sliced Wasserstein distance in applications. Previous works either utilize expensive optimization to select the slicing distribution or use slicing distributions that require expensive sampling methods. In this work, we propose an optimization-free slicing distribution that provides a fast sampling for the Monte Carlo estimation of expectation. In particular, we introduce the random-path projecting direction (RPD) which is constructed by leveraging the normalized difference between two random vectors following the two input measures. From the RPD, we derive the random-path slicing distribution (RPSD) and two variants of sliced Wasserstein, i.e., the Random-Path Projection Sliced Wasserstein (RPSW) and the Importance Weighted Random-Path Projection Sliced Wasserstein (IWRPSW). We then discuss the topological, statistical, and computational properties of RPSW and IWRPSW. Finally, we showcase the favorable performance of RPSW and IWRPSW in gradient flow and the training of denoising diffusion generative models on images.

Sliced Wasserstein with Random-Path Projecting Directions

TL;DR

The paper tackles the inefficiency of selecting slicing directions in Sliced Wasserstein distances by introducing Random-Path Projecting Direction (RPD) and Random-Path Slicing Distribution (RPSD), enabling optimization-free sampling. It defines two SW variants, Random-Path Projection Sliced Wasserstein (RPSW) and Importance Weighted RPSW (IWRPSW), and establishes their theoretical properties, including topological and statistical aspects and favorable computational complexity. The authors demonstrate that RPSW and IWRPSW outperform traditional SW variants in gradient-flow tasks and in training denoising diffusion models, achieving faster convergence and improved generation quality with reduced compute. This approach offers a scalable, discriminative, and practical alternative to optimization-based slicing schemes, with potential extensions to manifolds and a wide range of downstream applications in generative modeling and beyond.

Abstract

Slicing distribution selection has been used as an effective technique to improve the performance of parameter estimators based on minimizing sliced Wasserstein distance in applications. Previous works either utilize expensive optimization to select the slicing distribution or use slicing distributions that require expensive sampling methods. In this work, we propose an optimization-free slicing distribution that provides a fast sampling for the Monte Carlo estimation of expectation. In particular, we introduce the random-path projecting direction (RPD) which is constructed by leveraging the normalized difference between two random vectors following the two input measures. From the RPD, we derive the random-path slicing distribution (RPSD) and two variants of sliced Wasserstein, i.e., the Random-Path Projection Sliced Wasserstein (RPSW) and the Importance Weighted Random-Path Projection Sliced Wasserstein (IWRPSW). We then discuss the topological, statistical, and computational properties of RPSW and IWRPSW. Finally, we showcase the favorable performance of RPSW and IWRPSW in gradient flow and the training of denoising diffusion generative models on images.
Paper Structure (19 sections, 5 theorems, 54 equations, 5 figures, 2 tables, 3 algorithms)

This paper contains 19 sections, 5 theorems, 54 equations, 5 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Random-Path Projection Sliced Wasserstein
  4. Random-Path Projecting Direction
  5. Random-Path Projection Sliced Wasserstein
  6. Experiments
  7. Toy Gradient Flows
  8. Gradient Flows on Images
  9. Denoising Diffusion Models
  10. Conclusion
  11. Proofs
  12. Proof of Theorem \ref{['theo:metricity']}
  13. Proof of Proposition \ref{['prop:connection']}
  14. Proof of Proposition \ref{['prop:sample_complexity']}
  15. Proof of Proposition \ref{['proposition:MCerror']}
  16. ...and 4 more sections

Key Result

Theorem 1

For any $p \geq 1$, $L \geq 1$, $f:[0,\infty) \to (0,\infty)$ and $0<\kappa<\infty$, the random-path projection sliced Wasserstein $\text{RPSW}_{p}(\cdot,\cdot;\sigma_\kappa)$ and the importance weighted random-path projection sliced Wasserstein $\text{IWRPSW}_{p}(\cdot,\cdot;\sigma_\kappa,L)$ are s where we have $\text{RPSW}_{p}^p(\mu_1,\mu_3;\sigma_\kappa,\mu_1,\mu_2) = \mathbb{E}_{\theta \sim \

Figures (5)

  • Figure 1: Results for gradient flows that are from the empirical distribution over the color points to the empirical distribution over S-shape points produced by different SW variants. The corresponding Wasserstein-2 distance between the empirical distribution at the current step and the S-shape distribution and the computational time (in second) to reach the step is reported at the top of the figure.
  • Figure 2: Gradient flows from MNIST digit 1 to MNIST digit 0. .
  • Figure 3: Random generated images on CIFAR10 from DDGAN, RPSW-DD, and IWRPSW-DD.
  • Figure 4: Results for gradient flows that are from the empirical distribution over the color points to the empirical distribution over S-shape points produced by different RPSW and IWRPSW with original gradient estimator.
  • Figure 5: Gradient flows from MNIST digit 1 to MNIST digit 0 .

Theorems & Definitions (16)

  • Definition 1: Random-path
  • Definition 2: Random-path projecting direction
  • Remark 1
  • Definition 3
  • Remark 2
  • Definition 4
  • Remark 3
  • Definition 5
  • Theorem 1
  • Remark 4
  • ...and 6 more