Gaussian-Smoothed Sliced Probability Divergences

TL;DR

This work studies Gaussian-smoothed sliced divergences as privacy-preserving tools for distribution comparison. It establishes that smoothing and slicing preserve metric properties and the weak topology, and introduces a double empirical distribution to analyze statistical behavior, yielding an $O(n^{-1/2})$ convergence rate for Gaussian-smoothed sliced Wasserstein under mild conditions. The authors also derive continuity and noise-level dependency results, enabling cheap warm-starts for privacy-utility tuning, and validate the theory with domain-adaptation experiments showing little performance loss under privacy constraints. Practically, the approach offers dimension-free sample complexity and flexible privacy-utility trade-offs, with broad applicability to privacy-preserving distribution comparison beyond SW to general divergences.

Abstract

Gaussian smoothed sliced Wasserstein distance has been recently introduced for comparing probability distributions, while preserving privacy on the data. It has been shown that it provides performances similar to its non-smoothed (non-private) counterpart. However, the computationaland statistical properties of such a metric have not yet been well-established. This work investigates the theoretical properties of this distance as well as those of generalized versions denoted as Gaussian-smoothed sliced divergences. We first show that smoothing and slicing preserve the metric property and the weak topology. To study the sample complexity of such divergences, we then introduce $\hat{\hatμ}_{n}$ the double empirical distribution for the smoothed-projected $μ$. The distribution $\hat{\hatμ}_{n}$ is a result of a double sampling process: one from sampling according to the origin distribution $μ$ and the second according to the convolution of the projection of $μ$ on the unit sphere and the Gaussian smoothing. We particularly focus on the Gaussian smoothed sliced Wasserstein distance and prove that it converges with a rate $O(n^{-1/2})$. We also derive other properties, including continuity, of different divergences with respect to the smoothing parameter. We support our theoretical findings with empirical studies in the context of privacy-preserving domain adaptation.

Figures (7)

• Figure 1: Measuring the divergence between two sets of samples in $\mathbb{R}^{50}$, of increasing size, randomly drawn from $\mathcal{N}(0,\mathbf{I})$. We compare three sliced divergences and their Gaussian-smoothed sliced versions with a $\sigma=3$: (top) dimension has been set to $d=50$; (bottom) sample complexity with different dimensions. This plot confirms that the complexity is dimension-independent.
• Figure 2: Absolute difference between the approximated Monte Carlo approximation of all divergences compared to the true one (evaluated with $10,000$ number of projections). The two sets of $500$ samples in $\mathbb{R}^{50}$ are randomly drawn from $\mathcal{N}(0,\mathbf{I})$. The Gaussian-smoothed sliced divergences are parameterized with $\sigma=3$.
• Figure 3: Measuring the divergence between two sets of samples in $\mathbb{R}^{50}$ drawn from $\mathcal{N}(0,\mathbf{I})$. We plot the sample complexity for different Gaussian-smoothed sliced divergence at different level of noises.
• Figure 4: Domain adaptation performances using different divergences on distributions with respect to the Gaussian smoothing. (Left) USPS to MNIST. (Middle) Office-31 Webcam to DSLR. (Right) Office-31 Amazon to Webcam.
• Figure 5: Domain adaptation performances using different divergences on distributions with respect to the Gaussian smoothing using one-epoch-fine-tuned models. (Left) USPS to MNIST. (Middle) Office-31 Webcam to DSLR. (Right) Office-31 Amazon to Webcam.

Theorems & Definitions (22)

• Definition 1
• Definition 2
• Definition 3
• Theorem 1
• Theorem 2
• Proposition 1
• Remark 1
• Lemma 1
• Definition 4
• Remark 2