Kolmogorov-Smirnov GAN

TL;DR

KSGAN introduces a Generalized Kolmogorov–Smirnov distance objective for training deep implicit generative models, replacing traditional divergences with a metric-based criterion derived from multidimensional KS theory. The framework uses neural level-set critics to implicitly model generalized quantiles and optimizes a generator to minimize $D_{\mathrm{GKS}}$, aided by an energy-based critic and auxiliary objectives. Empirical results on synthetic data, MNIST, and CIFAR-10 demonstrate competitive performance and improved training stability, with reduced sensitivity to hyperparameters and lower variance across initializations. The work connects KS-based distribution metrics to adversarial learning, offering a principled alternative to Wasserstein- or MMD-based GAN variants and suggesting avenues for future exploration of generalized quantile representations and critic design.

Abstract

We propose a novel deep generative model, the Kolmogorov-Smirnov Generative Adversarial Network (KSGAN). Unlike existing approaches, KSGAN formulates the learning process as a minimization of the Kolmogorov-Smirnov (KS) distance, generalized to handle multivariate distributions. This distance is calculated using the quantile function, which acts as the critic in the adversarial training process. We formally demonstrate that minimizing the KS distance leads to the trained approximate distribution aligning with the target distribution. We propose an efficient implementation and evaluate its effectiveness through experiments. The results show that KSGAN performs on par with existing adversarial methods, exhibiting stability during training, resistance to mode dropping and collapse, and tolerance to variations in hyperparameter settings. Additionally, we review the literature on the Generalized KS test and discuss the connections between KSGAN and existing adversarial generative models.

Figures (5)

• Figure 1: A schematic depiction of how the Generalized Kolmogorov-Smirnov (KS) distance between target$\mathds{P}_{F}$ and approximate$\mathds{P}_{G}$ distributions with respect to critic$c_{\phi}$ is computed. The critic is evaluated on samples $x_{F}$ ($\color{target_color}\pmb{|}$) and $x_{G}$ ($\color{approx_color}\pmb{|}$) from the target and approximate distributions respectively. The $\lambda$threshold moves from $-\infty$ to $+\infty$ establishing a stack of level sets. At each level, the fraction of datapoints ($\color{target_color}\bullet$ and $\color{approx_color}\bullet$) below the threshold is calculated for each distribution independently. This produces the $\mathds{P}_{F}\left(\Gamma_{c_{\phi}}(\lambda)\right)$ and $\mathds{P}_{G}\left(\Gamma_{c_{\phi}}(\lambda)\right)$ curves. The Generalized KS distance is the largest absolute difference between the curves shown as $\color{ks_color}\pmb{\updownarrow}$ in the right figure. Best viewed in color.
• Figure 2: Squared population MMD between approximate and test distribution as a function of the number of training instances. Solid lines denote the average over five random initializations, and the shaded area represents the two-$\sigma$ interval. Best viewed in color.
• Figure 3: Histograms of samples from distributions denoted on the top. Heatmap colors are shared for all figures in each row. Best viewed in color.
• Figure 4: Samples from the respective models trained on the MNIST dataset.
• Figure 5: Samples from the respective models trained on the CIFAR-10 dataset. Best viewed in color.

Theorems & Definitions (6)

• Definition 1: Generalized Quantile Function
• Example 1: Polonik1999
• Definition 2: Generalized Kolmogorov-Smirnov distance
• Remark 1: The silhouette Polonik1998
• Remark 2
• Definition 3: Neural level set