Bayesian Neural Networks: A Min-Max Game Framework

TL;DR

This work addresses robustness and uncertainty quantification in neural networks by applying a minimax framework to Bayesian neural networks. It introduces a MinMax Bayesian Neural Network with a two-player game between a deterministic center f and a stochastic equator g = f + r xi, linking the minimax loss to a representation-based objective and controlling variance through a radius r. The authors derive a practical formulation, compare it to classical BNNs, and validate on MNIST, FMNIST, and CIFAR-10, analyzing radius search, BN effects, and noise perturbations. The framework offers a conservative, adjustable-variance alternative with potential for out-of-distribution detection, while highlighting methodological and optimization challenges such as radius search reliability and BN compatibility.

Abstract

In deep learning, Bayesian neural networks (BNN) provide the role of robustness analysis, and the minimax method is used to be a conservative choice in the traditional Bayesian field. In this paper, we study a conservative BNN with the minimax method and formulate a two-player game between a deterministic neural network $f$ and a sampling stochastic neural network $f + r*ξ$. From this perspective, we understand the closed-loop neural networks with the minimax loss and reveal their connection to the BNN. We test the models on simple data sets, study their robustness under noise perturbation, and report some issues for searching $r$.

Figures (7)

• Figure : (a) Normal case, concave function
• Figure : (a) CNN without bias or BN, 128 dim
• Figure : (a) CNN without bias or BN, 11 dim
• Figure : (a) Golden search, $log$
• Figure : (a) Golden search, $log$