Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MIQCQP reformulation of the ReLU neural networks Lipschitz constant estimation problem

Mohammed Sbihi, Sophie Jan, Nicolas Couellan

TL;DR

By taking into account activation regions at each layer as new constraints, new quadratically constrained MIP formulations for the neural network Lipschitz estimation problem are proposed.

Abstract

It is well established that to ensure or certify the robustness of a neural network, its Lipschitz constant plays a prominent role. However, its calculation is NP-hard. In this note, by taking into account activation regions at each layer as new constraints, we propose new quadratically constrained MIP formulations for the neural network Lipschitz estimation problem. The solutions of these problems give lower bounds and upper bounds of the Lipschitz constant and we detail conditions when they coincide with the exact Lipschitz constant.

MIQCQP reformulation of the ReLU neural networks Lipschitz constant estimation problem

TL;DR

By taking into account activation regions at each layer as new constraints, new quadratically constrained MIP formulations for the neural network Lipschitz estimation problem are proposed.

Abstract

It is well established that to ensure or certify the robustness of a neural network, its Lipschitz constant plays a prominent role. However, its calculation is NP-hard. In this note, by taking into account activation regions at each layer as new constraints, we propose new quadratically constrained MIP formulations for the neural network Lipschitz estimation problem. The solutions of these problems give lower bounds and upper bounds of the Lipschitz constant and we detail conditions when they coincide with the exact Lipschitz constant.
Paper Structure (7 sections, 4 theorems, 31 equations)

This paper contains 7 sections, 4 theorems, 31 equations.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Deriving lower and upper bounds of the Lipschitz constant of ReLU networks
  4. MIQCQP reformulations
  5. The case $p=2$
  6. The case $p=1$
  7. The case $p=\infty$

Key Result

Proposition 1

For each $k\in \{1,\cdots,L-1\}$ and each $i\in\{1,\dots, n_k\}$, the function $g_k^i \mapsto N(g)$ is convex.

Theorems & Definitions (9)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Remark 1
  • Corollary 1
  • proof