BoxRL-NNV: Boxed Refinement of Latin Hypercube Samples for Neural Network Verification

TL;DR

The paper introduces BoxRL-NNV, a Python-based verification tool that bounds neural network outputs given input ranges from VNNLIB to detect safety violations. The method combines Latin Hypercube Sampling with a multidimensional uniformity variant (LHSMDU) for global extrema estimation and limited-memory BFGS-B (L-BFGS-B) for local refinement, followed by Z3 SMT-based analysis. Applied to the ACAS Xu benchmark, BoxRL-NNV demonstrates feasible verification across 50 evaluations, yielding holds, violations, and unknown results within a few minutes. While effective in practice, the approach highlights scalability challenges for high-dimensional inputs and motivates future work on surrogate modeling to derive interpretable safety rules.

Abstract

BoxRL-NNV is a Python tool for the detection of safety violations in neural networks by computing the bounds of the output variables, given the bounds of the input variables of the network. This is done using global extrema estimation via Latin Hypercube Sampling, and further refinement using L-BFGS-B for local optimization around the initial guess. This paper presents an overview of BoxRL-NNV, as well as our results for a subset of the ACAS Xu benchmark. A complete evaluation of the tool's performance, including benchmark comparisons with state-of-the-art tools, shall be presented at the Sixth International Verification of Neural Networks Competition (VNN-COMP'25).