BOLT-GAN: Bayes-Optimal Loss for Stable GAN Training

TL;DR

BOLT-GAN introduces a Bayes-optimal learning threshold loss to train GAN discriminators, aligning them with the Bayes optimal classifier and guiding generators to reduce Bayes error. The authors show that, in the vanilla form, the BOLT objective effectively minimizes total variation, which can cause instability, and they remedy this by enforcing a 1-Lipschitz constraint on the discriminator, linking the regularized objective to the Wasserstein distance for controlled convergence. They provide a convergence analysis connecting the BG objective to TV and demonstrate that Lipschitz BOLT-GAN yields stable gradient signals while still driving the generator toward the data distribution. Empirically, Lipschitz BOLT-GAN achieves 10–60% improvements in Frechet Inception Distance (FID) across CIFAR-10, CelebA-64, and LSUN datasets, validating its stability and effectiveness for high-quality image synthesis.

Abstract

We introduce BOLT-GAN, a simple yet effective modification of the WGAN framework inspired by the Bayes Optimal Learning Threshold (BOLT). We show that with a Lipschitz continuous discriminator, BOLT-GAN implicitly minimizes a different metric distance than the Earth Mover (Wasserstein) distance and achieves better training stability. Empirical evaluations on four standard image generation benchmarks (CIFAR-10, CelebA-64, LSUN Bedroom-64, and LSUN Church-64) show that BOLT-GAN consistently outperforms WGAN, achieving 10-60% lower Frechet Inception Distance (FID). Our results suggest that BOLT is a broadly applicable principle for enhancing GAN training.

Figures (5)

• Figure 1: CIFAR-10 FID vs. epochs (mean$\pm$std, 3 seeds).
• Figure 2: CIFAR-10 samples at 100 epochs.
• Figure 3: CIFAR-10 FID vs. epochs for different $\lambda_{\mathrm{GP}}$ (mean$\pm$std, 3 seeds).
• Figure 4: BOLT--GAN--GP on CelebA--64: FID vs. epochs for different $\lambda_{\mathrm{GP}}$ values (mean$\pm$std over 3 seeds). The trend mirrors CIFAR-10: near-optimal, stable performance for $\lambda_{\mathrm{GP}}\!\in\![5, 10]$.
• Figure 5: Qualitative samples (first experiment of the main paper). BOLT-GAN-GP after 20 epochs with $\lambda_{\mathrm{GP}}{=}10$. Top-left: CIFAR-10; top-right: CelebA-64; bottom-left: LSUN Bedroom-64; bottom-right: LSUN Church-64. Each panel shows the same $4\times4$ grid of generated samples at a reduced scale. Under the identical setup used in our main experiments, Lipschitz BOLT--GAN produces sharper textures, fewer checkerboard artifacts, and more stable color statistics than the WGAN--GP baseline.

Theorems & Definitions (47)

• Theorem 1: Binary Bayes error tavasoli2025universal
• Remark 1
• Remark 2
• Theorem 2: BOLT Estimator of MAP Classifier
• proof
• Remark 3: BOLT Bias and Variance
• Definition 1: Total variation norm
• Lemma 1: Dudley_2002
• Theorem 3: BOLT vs TV
• proof