Concave Certificates: Geometric Framework for Distributionally Robust Risk and Complexity Analysis
Hong T. M. Chu
TL;DR
This paper tackles distributionally robust optimization under Wasserstein uncertainty by developing a geometric framework built on least concave majorants of the loss growth rate. The core idea is a concave certificate that tightly bounds the distributionally robust risk $\,\mathcal{R}_p(\epsilon)$ without requiring convexity, differentiability, Lipschitzness, or bounded domains, and it extends to a deterministic generalization bound via a new concave complexity measure $\hat{\mathfrak{C}}_{\mathcal{Z}_N}(\mathcal{L},\epsilon)$. Key contributions include (i) a sandwich bound $\,\text{lb}_p(\epsilon) \le \mathcal{R}_p(\epsilon) - \hat{\mathcal{R}} \le \ \text{cc}_p(\epsilon)$, (ii) the introduction of concave complexity with contraction-like properties and dimension-free ARC/ACC gaps, (iii) a practical Adversarial Score to enable layer-wise robustness analysis in deep networks, and (iv) empirical validation on traffic regression and MNIST showing tighter certificates than traditional Lipschitz or gradient-based methods. The framework yields finite, dimension-free certificates even for non-Lipschitz objectives and provides a practical pathway to certify robustness in deep learning pipelines under distributional shifts. Overall, the concave certificate approach offers a rigorous, scalable tool for DR risk estimation and robustness analysis with concrete computational benefits. $ $
Abstract
Distributionally Robust (DR) optimization aims to certify worst-case risk within a Wasserstein uncertainty set. Current certifications typically rely either on global Lipschitz bounds, which are often conservative, or on local gradient information, which provides only a first-order approximation. This paper introduces a novel geometric framework based on the least concave majorants of the growth rate function. Our proposed concave certificate establishes a tight bound of DR risk that remains applicable to non-Lipschitz and non-differentiable losses. We extend this framework to complexity analysis, introducing a deterministic bound that complements standard statistical generalization bound. Furthermore, we utilize this certificate to bound the gap between adversarial and empirical Rademacher complexity, demonstrating that dependencies on input diameter, network width, and depth can be eliminated. For practical application in deep learning, we introduce the adversarial score as a tractable relaxation of the concave certificate that enables efficient and layer-wise analysis of neural networks. We validate our theoretical results in various numerical experiments on classification and regression tasks on real-world data.