Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computing AD-compatible subgradients of convex relaxations of implicit functions

Yingkai Song, Kamil A. Khan

TL;DR

This work introduces the first subgradient propagation rules for implicit-function convex relaxations constructed as optimal-value problems, enabling their integration into forward AD workflows. By distinguishing cases based on the dimensions of the implicit variables $n_x$ and parameters $n_p$, the authors develop closed-form subgradients for $n_x=1$, LP-based directional derivatives for small $n_p$, and lexicographic derivatives for larger $n_p$, all while maintaining compatibility with existing McCormick-style relaxations. The approach leverages residual relaxations, piecewise differentiable relaxations, and Slater-type conditions to derive practical subgradient evaluations, demonstrated through proof-of-concept Julia implementations on thermodynamic and reactor-design problems. The results enable AD-compatible inclusion of implicit-function relaxations in global optimization pipelines, potentially improving lower-bound computations and sensitivity analyses in a wide range of applications. Overall, the paper advances the toolkit for nonsmooth optimization by bridging implicit function relaxations with forward-mode AD and subgradient propagation.

Abstract

Automatic generation of convex relaxations and subgradients is critical in global optimization, and is typically carried out using variants of automatic/algorithmic differentiation (AD). At previous AD conferences, variants of the forward and reverse AD modes were presented to evaluate accurate subgradients for convex relaxations of supplied composite functions. In a recent approach for generating convex relaxations of implicit functions, these relaxations are constructed as optimal-value functions; this formulation is versatile but complicates sensitivity analysis. We present the first subgradient propagation rules for these implicit function relaxations, based on supplied AD-like knowledge of the residual function. Our new subgradient rules allow implicit function relaxations to be added to the elemental function libraries for the forward AD modes for subgradient propagation of convex relaxations. Proof-of-concept numerical results in Julia are presented.

Computing AD-compatible subgradients of convex relaxations of implicit functions

TL;DR

This work introduces the first subgradient propagation rules for implicit-function convex relaxations constructed as optimal-value problems, enabling their integration into forward AD workflows. By distinguishing cases based on the dimensions of the implicit variables and parameters , the authors develop closed-form subgradients for , LP-based directional derivatives for small , and lexicographic derivatives for larger , all while maintaining compatibility with existing McCormick-style relaxations. The approach leverages residual relaxations, piecewise differentiable relaxations, and Slater-type conditions to derive practical subgradient evaluations, demonstrated through proof-of-concept Julia implementations on thermodynamic and reactor-design problems. The results enable AD-compatible inclusion of implicit-function relaxations in global optimization pipelines, potentially improving lower-bound computations and sensitivity analyses in a wide range of applications. Overall, the paper advances the toolkit for nonsmooth optimization by bridging implicit function relaxations with forward-mode AD and subgradient propagation.

Abstract

Automatic generation of convex relaxations and subgradients is critical in global optimization, and is typically carried out using variants of automatic/algorithmic differentiation (AD). At previous AD conferences, variants of the forward and reverse AD modes were presented to evaluate accurate subgradients for convex relaxations of supplied composite functions. In a recent approach for generating convex relaxations of implicit functions, these relaxations are constructed as optimal-value functions; this formulation is versatile but complicates sensitivity analysis. We present the first subgradient propagation rules for these implicit function relaxations, based on supplied AD-like knowledge of the residual function. Our new subgradient rules allow implicit function relaxations to be added to the elemental function libraries for the forward AD modes for subgradient propagation of convex relaxations. Proof-of-concept numerical results in Julia are presented.

Paper Structure

This paper contains 14 sections, 6 theorems, 33 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. Convex relaxations and subgradients
  5. Implicit function relaxations
  6. Lexicographic differentiation
  7. Outline of new subgradient formulations
  8. Subgradients for ${n_x=1}$
  9. Subgradients for ${n_x>1}$
  10. The case in which ${n_p=1}$:
  11. The case in which ${n_p=2}$:
  12. The case in which ${n_p>2}$:
  13. Numerical examples
  14. Conclusions

Key Result

Proposition 2.1

Consider the setup in Definition Definition:implicit. Let $\mathbf{f}^\mathrm{cv},\mathbf{f}^\mathrm{cc}:X\times P\to\mathbb{R}^{n_x}$ be convex and concave relaxations of $\mathbf{f}$ on $X\times P$, respectively. Define $\mathbf{x}^\mathrm{cv},\mathbf{x}^\mathrm{cc}:P\to\bar{\mathbb{R}}^{n_x}$ suc If these optimization problems are infeasible, then set $x_i^\mathrm{cv}(\mathbf{p}):=+\infty$ and

Figures (3)

  • Figure 1: The implicit function $V$ (blue) in terms of $P$ and $T$ on $[0.5,1.1]\times[250,320]$, along with the constructed convex and concave relaxations (red), for Example \ref{['exp:1']}.
  • Figure 2: A cross-section at $p_1:=0.6$ of the implicit function $z_3$ (solid blue) of $(p_1,p_2)$, along with convex and concave relaxations (dashed red) and subtangents (dotted black) generated from the reference point $(p_1,p_2):=(0.6,1.348)$, for Example \ref{['exp:2']}.
  • Figure 3: A cross-section at $p_1:=0.40$ of the implicit function $z_1$ (blue) of $(p_1,p_2, p_3)$, along with convex and concave relaxations (red) and subtangents constructed using our new approach (black), for Example \ref{['exp:3']}. One subtangent is generated using the subgradient of the convex relaxation at $(p_1,p_2,p_3):= (0.40,0.0575,8.7 )$, and another is generated using the subgradient of the concave relaxation at $(p_1,p_2,p_3):= (0.40,0.0545,9.6 )$.

Theorems & Definitions (17)

  • Definition 2.1: from cao2023general
  • Definition 2.2: from scholtes2012introduction
  • Definition 2.3: from rockafellar2015convex
  • Definition 2.4: from cao2023general
  • Proposition 2.1: from cao2023general
  • Definition 2.5: from nesterov2005lexicographic
  • Definition 2.6: from KhanBartonFwdAD
  • Proposition 4.1
  • Proposition 5.1
  • Remark 5.1
  • ...and 7 more