Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Relu and softplus neural nets as zero-sum turn-based games

Stephane Gaubert, Yiannis Vlassopoulos

TL;DR

The paper reveals a precise, forward-looking connection between ReLU and Softplus neural networks and zero-sum game theory by representing network maps as values of backward, turn-based stopping games. It provides a discrete path-integral interpretation of network outputs via the Shapley-Bellman recursion and extends the framework to entropically regularized games that correspond to Softplus activations. The main contributions include a rigorous ReLU-net/game equivalence, a path-integral view with monotone bounds and certificate-based verification, and a principled inverse-game perspective on training, with Softplus nets arising as the entropy-regularized limit. This work offers a novel, theoretically grounded lens for neural network analysis, robustness certification, and potentially new training paradigms through game-theoretic formulations.

Abstract

We show that the output of a ReLU neural network can be interpreted as the value of a zero-sum, turn-based, stopping game, which we call the ReLU net game. The game runs in the direction opposite to that of the network, and the input of the network serves as the terminal reward of the game. In fact, evaluating the network is the same as running the Shapley-Bellman backward recursion for the value of the game. Using the expression of the value of the game as an expected total payoff with respect to the path measure induced by the transition probabilities and a pair of optimal policies, we derive a discrete Feynman-Kac-type path-integral formula for the network output. This game-theoretic representation can be used to derive bounds on the output from bounds on the input, leveraging the monotonicity of Shapley operators, and to verify robustness properties using policies as certificates. Moreover, training the neural network becomes an inverse game problem: given pairs of terminal rewards and corresponding values, one seeks transition probabilities and rewards of a game that reproduces them. Finally, we show that a similar approach applies to neural networks with Softplus activation functions, where the ReLU net game is replaced by its entropic regularization.

Relu and softplus neural nets as zero-sum turn-based games

TL;DR

The paper reveals a precise, forward-looking connection between ReLU and Softplus neural networks and zero-sum game theory by representing network maps as values of backward, turn-based stopping games. It provides a discrete path-integral interpretation of network outputs via the Shapley-Bellman recursion and extends the framework to entropically regularized games that correspond to Softplus activations. The main contributions include a rigorous ReLU-net/game equivalence, a path-integral view with monotone bounds and certificate-based verification, and a principled inverse-game perspective on training, with Softplus nets arising as the entropy-regularized limit. This work offers a novel, theoretically grounded lens for neural network analysis, robustness certification, and potentially new training paradigms through game-theoretic formulations.

Abstract

We show that the output of a ReLU neural network can be interpreted as the value of a zero-sum, turn-based, stopping game, which we call the ReLU net game. The game runs in the direction opposite to that of the network, and the input of the network serves as the terminal reward of the game. In fact, evaluating the network is the same as running the Shapley-Bellman backward recursion for the value of the game. Using the expression of the value of the game as an expected total payoff with respect to the path measure induced by the transition probabilities and a pair of optimal policies, we derive a discrete Feynman-Kac-type path-integral formula for the network output. This game-theoretic representation can be used to derive bounds on the output from bounds on the input, leveraging the monotonicity of Shapley operators, and to verify robustness properties using policies as certificates. Moreover, training the neural network becomes an inverse game problem: given pairs of terminal rewards and corresponding values, one seeks transition probabilities and rewards of a game that reproduces them. Finally, we show that a similar approach applies to neural networks with Softplus activation functions, where the ReLU net game is replaced by its entropic regularization.
Paper Structure (34 sections, 9 theorems, 121 equations, 2 figures, 2 tables)

This paper contains 34 sections, 9 theorems, 121 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Summary of results
  3. The ReLU net game
  4. The Softplus net game
  5. Related work
  6. Organization of the paper
  7. Acknowledgments
  8. Basics of Markov decision processes
  9. Basics of Repeated zero-sum Games
  10. Concurrent games
  11. Turn-based games
  12. Player Max's turn.
  13. Player Min's turn.
  14. Stopping games
  15. ReLU neural net as a turn-based, stopping game
  16. ...and 19 more sections

Key Result

Proposition 1

Given a ReLU neural net, consider the ReLU net game defined above, then the values of the game, $V_{i+}^l$ and $V_{i-}^l$, satisfy the following Shapley-Bellman equations: with boundary conditions $V^L_{i,+}(x)\coloneqq x_i$ and $V^L_{i,-}(x)\coloneqq-x_i$ where the vector $x\coloneqq(x_1,\dots,x_{n_1})$ is the input to the network.

Figures (2)

  • Figure 1: Graph of the game corresponding to the ReLU neural net in the example in Section 4.3. The circles denote the states. A diamond after a state denotes the 2 possible actions at the state: stop (and get 0 reward) or continue and get the reward denoted on the edge exiting the diamond. A square is the transition to the next state. The edges exiting a square denote the non-trivial choices and the transition probabilities are indicated along these edges. The arrows point at the direction the game is played which is the opposite of the one the neural net is running. Therefore the inputs $(x_1,x_2)$ to the net are the terminal rewards of the game. The evaluation of the neural net coincides with the Shapley-Bellman backward recursion for the value of the game.
  • Figure 2: The grid indicates the neurons in a ReLU net. Each neuron $(l,i)$ corresponds to 2 game states, one $(l,i+)$ where Max plays and one $(l,i-)$ where Min plays, but we don't indicate this in the figure so as not to clutter it. A given input $x\coloneqq(x_1,\dots,x_{15})$ to the ReLU net is interpreted as the terminal reward of the ReLU net game. The corresponding optimal policies $\bm{\pi}^*(x):S^+ \to \{0,1\}$ and $\bm{\sigma}^*(x): S^- \to \{0,1\}$ determine 2 Boolean patterns on the vertices (one for the Max labeled states and one for the Min, (which is exactly the opposite: see Prop. \ref{['optimal pol']}). Game paths (drawn in blue) contributing to the value of the ReLU net game (which is equal to the output of the ReLU net) for the given $x$, start at the bottom row and proceed through $1$-labeled vertices, ending either at a 0-labeled vertex before reaching the top or when they reach the top. A plus sign on an edge indicates that the corresponding weight is positive and therefore the same player keep playing. A minus sign indicates the corresponding weight is negative and therefore the player changes. The sum over paths \ref{['Vop']} gives the value of the game which is equal to the output of the neural net.

Theorems & Definitions (31)

  • Definition 1
  • Definition 2: Finite-Horizon Zero–Sum Game
  • Remark 1
  • Proposition 1
  • proof
  • Remark 2
  • Theorem 1
  • proof
  • Remark 3
  • Proposition 2: Lipschitz constant of the ReLU net map
  • ...and 21 more