Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Failure of uniform laws of large numbers for subdifferentials and beyond

Lai Tian, Johannes O. Royset

TL;DR

The paper addresses whether uniform laws of large numbers for subdifferentials extend to nonsmooth random functions. It demonstrates negative results for random Lipschitz functions and, in higher dimensions, for random convex functions, by constructing probabilistic gadgets that maintain a persistent discrepancy between empirical and true subgradients; it also shows a positive, univariate convex case where a uniform LLN with $r=0$ holds. The results reveal a fundamental separation between uniform and graphical LLNs for set-valued mappings and imply limitations for sample-average approximation in Stochastic Programming and empirical risk minimization when nonsmoothness is present. The work highlights the central role of dimensionality and nonsmooth structure in governing the behavior of subdifferential LLNs and motivates the use of weaker notions of convergence (graphical LLN) or alternative frameworks for robust optimization and learning.

Abstract

We provide counterexamples showing that uniform laws of large numbers do not hold for subdifferentials under natural assumptions. Our results apply to random Lipschitz functions and random convex functions with a finite number of smooth pieces. Consequently, they resolve the questions posed by Shapiro and Xu [J. Math. Anal. Appl., 325(2), 2007] in the negative and highlight the obstacles nonsmoothness poses to uniform results.

Failure of uniform laws of large numbers for subdifferentials and beyond

TL;DR

The paper addresses whether uniform laws of large numbers for subdifferentials extend to nonsmooth random functions. It demonstrates negative results for random Lipschitz functions and, in higher dimensions, for random convex functions, by constructing probabilistic gadgets that maintain a persistent discrepancy between empirical and true subgradients; it also shows a positive, univariate convex case where a uniform LLN with holds. The results reveal a fundamental separation between uniform and graphical LLNs for set-valued mappings and imply limitations for sample-average approximation in Stochastic Programming and empirical risk minimization when nonsmoothness is present. The work highlights the central role of dimensionality and nonsmooth structure in governing the behavior of subdifferential LLNs and motivates the use of weaker notions of convergence (graphical LLN) or alternative frameworks for robust optimization and learning.

Abstract

We provide counterexamples showing that uniform laws of large numbers do not hold for subdifferentials under natural assumptions. Our results apply to random Lipschitz functions and random convex functions with a finite number of smooth pieces. Consequently, they resolve the questions posed by Shapiro and Xu [J. Math. Anal. Appl., 325(2), 2007] in the negative and highlight the obstacles nonsmoothness poses to uniform results.

Paper Structure

This paper contains 24 sections, 8 theorems, 45 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Notation.
  3. Main Results
  4. Random Lipschitz Functions
  5. Random Convex Functions
  6. Discussion
  7. Proofs
  8. A Probabilistic Gadget
  9. Proof of \ref{['thm:hard-lip']}
  10. Intuition.
  11. Step 1.
  12. Step 2.
  13. Step 3.
  14. Step 4.
  15. Step 5.
  16. ...and 9 more sections

Key Result

Theorem 1

Let $X=[0,1]\subset \mathbb{R}$ and $\bm{\xi}^1,\bm{\xi}^2,\ldots$ be iid random variables on the complete probability space $(\Omega,\mathcal{F},\mathbb{P})$, each uniformly distributed on $\Xi=[0,1]$. There exist a Carathéodory $f:\Xi\times \mathbb{R} \to \mathbb{R}$ and $\delta^\nu \in (0,1]\down

Figures (2)

  • Figure 1: The function $f(\xi,\cdot)$ in \ref{['sec:proof-lip']} when $\xi=0.01010101001_2$.
  • Figure 2: The functions $g(\xi,(0,\cdot))$ and $\max\{g(\xi,\cdot),0\}$ in \ref{['sec:proof-cvx']} when $\xi=0.01010101001_2$.

Theorems & Definitions (12)

  • Theorem 1: random Lipschitz
  • Corollary 1: subdifferential
  • proof
  • Theorem 2: cf. shapiro1996convergence and shapiro2021lectures
  • Proposition 1: univariate random convex
  • Theorem 3: random convex
  • Corollary 2: $\varepsilon$-subdifferential
  • proof
  • Lemma 1
  • proof
  • ...and 2 more