HiQ-Lip: A Hierarchical Quantum-Classical Method for Global Lipschitz Constant Estimation of ReLU Networks

TL;DR

HiQ-Lip introduces a quantum-classical hybrid framework to estimate the global Lipschitz constant of neural networks by converting the problem to a QUBO and applying a multilevel graph coarsening/refinement strategy compatible with small-scale quantum devices. The method achieves comparable upper-bound accuracy to state-of-the-art SDP-based approaches while significantly reducing computation time, confirmed on two-layer and multi-layer fully connected networks. Extensions to deep ReLU networks use layer-wise recursion, block-wise bounds, and depth-adaptive damping to maintain tractability and tighten estimates, with results showing up to 120× speedups over baselines. This work demonstrates the practical potential of using Coherent Ising Machines and similar quantum accelerators for neural network robustness evaluations, paving the way for scalable quantum-assisted analyses of larger architectures.

Abstract

Estimating the global Lipschitz constant of neural networks is crucial for understanding and improving their robustness and generalization capabilities. However, precise calculations are NP-hard, and current semidefinite programming (SDP) methods face challenges such as high memory usage and slow processing speeds. In this paper, we propose HiQ-Lip, a hybrid quantum-classical hierarchical method that leverages quantum computing to estimate the global Lipschitz constant. We tackle the estimation by converting it into a Quadratic Unconstrained Binary Optimization problem and implement a multilevel graph coarsening and refinement strategy to adapt to the constraints of contemporary quantum hardware. Our experimental evaluations on fully connected neural networks demonstrate that HiQ-Lip not only provides estimates comparable to state-of-the-art methods but also significantly accelerates the computation process. In specific tests involving two-layer neural networks with 256 hidden neurons, HiQ-Lip doubles the solving speed and offers more accurate upper bounds than the existing best method, LiPopt. These findings highlight the promising utility of small-scale quantum devices in advancing the estimation of neural network robustness.