Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Legendre Transform

Aleksey Minabutdinov, Patrick Cheridito

TL;DR

The paper introduces Deep Legendre Transform (DLT), a neural-network framework for approximating convex conjugates via an implicit Fenchel formulation, scalable to high dimensions and accompanied by a posteriori error estimates. By learning a function gθ on the gradient image D = ∇f(C) to satisfy f*(∇f(x)) = ⟨x, ∇f(x)⟩ − f(x), it enables high-dimensional computation of f* without requiring closed-form duals. The approach supports various NN architectures (ICNNs, ResNets, MLPs, KANs), enables error certification, and includes sampling strategies in gradient space to control distributional coverage; it also extends to time-dependent Hamilton–Jacobi problems (Time-DLT) and symbolic regression of conjugates (KANs). Empirical results show near-parity with direct dual-learning when f* is known, improved performance over grid methods in high dimensions, and exact symbolic recovery in select cases, highlighting practical impact for optimization, control, and PDE applications. Limitations include the need for differentiable convex f and confinement to D = ∇f(C); future work targets non-differentiable cases and broader domain coverage.

Abstract

We introduce a novel deep learning algorithm for computing convex conjugates of differentiable convex functions, a fundamental operation in convex analysis with various applications in different fields such as optimization, control theory, physics and economics. While traditional numerical methods suffer from the curse of dimensionality and become computationally intractable in high dimensions, more recent neural network-based approaches scale better, but have mostly been studied with the aim of solving optimal transport problems and require the solution of complicated optimization or max-min problems. Using an implicit Fenchel formulation of convex conjugation, our approach facilitates an efficient gradient-based framework for the minimization of approximation errors and, as a byproduct, also provides a posteriori error estimates for the approximation quality. Numerical experiments demonstrate our method's ability to deliver accurate results across different high-dimensional examples. Moreover, by employing symbolic regression with Kolmogorov--Arnold networks, it is able to obtain the exact convex conjugates of specific convex functions.

Deep Legendre Transform

TL;DR

The paper introduces Deep Legendre Transform (DLT), a neural-network framework for approximating convex conjugates via an implicit Fenchel formulation, scalable to high dimensions and accompanied by a posteriori error estimates. By learning a function gθ on the gradient image D = ∇f(C) to satisfy f*(∇f(x)) = ⟨x, ∇f(x)⟩ − f(x), it enables high-dimensional computation of f* without requiring closed-form duals. The approach supports various NN architectures (ICNNs, ResNets, MLPs, KANs), enables error certification, and includes sampling strategies in gradient space to control distributional coverage; it also extends to time-dependent Hamilton–Jacobi problems (Time-DLT) and symbolic regression of conjugates (KANs). Empirical results show near-parity with direct dual-learning when f* is known, improved performance over grid methods in high dimensions, and exact symbolic recovery in select cases, highlighting practical impact for optimization, control, and PDE applications. Limitations include the need for differentiable convex f and confinement to D = ∇f(C); future work targets non-differentiable cases and broader domain coverage.

Abstract

We introduce a novel deep learning algorithm for computing convex conjugates of differentiable convex functions, a fundamental operation in convex analysis with various applications in different fields such as optimization, control theory, physics and economics. While traditional numerical methods suffer from the curse of dimensionality and become computationally intractable in high dimensions, more recent neural network-based approaches scale better, but have mostly been studied with the aim of solving optimal transport problems and require the solution of complicated optimization or max-min problems. Using an implicit Fenchel formulation of convex conjugation, our approach facilitates an efficient gradient-based framework for the minimization of approximation errors and, as a byproduct, also provides a posteriori error estimates for the approximation quality. Numerical experiments demonstrate our method's ability to deliver accurate results across different high-dimensional examples. Moreover, by employing symbolic regression with Kolmogorov--Arnold networks, it is able to obtain the exact convex conjugates of specific convex functions.
Paper Structure (54 sections, 1 theorem, 53 equations, 5 figures, 21 tables)

This paper contains 54 sections, 1 theorem, 53 equations, 5 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Convex conjugation as a machine learning problem
  3. A posteriori approximation error estimates
  4. Sampling from a given distribution in gradient space
  5. Numerical experiments
  6. Numerical approximations with MLPs, ResNets, ICNNs
  7. Functions without closed form conjugate.
  8. Comparison with classical grid methods
  9. Direct vs. inverse sampling: distorting gradient mapping
  10. Applications to Hamilton--Jacobi equations
  11. Experimental results.
  12. Symbolic regression with KANs
  13. Conclusion, limitation and future work
  14. Unbiased a posteriori error estimator
  15. Technical appendix: additional experiments
  16. ...and 39 more sections

Key Result

Lemma 1

Let $f:C\to\mathbb{R}$ be differentiable and convex on an open convex set $C\subseteq\mathbb{R}^d$, and let $D=\nabla f(C)$. Fix a probability measure $\mu$ on $C$ and set $\nu:=\mu\circ(\nabla f)^{-1}$ on $D$. For a measurable $g:D\to\mathbb{R}$, assume Let $X_1,\dots,X_n\stackrel{\text{i.i.d.}}{\sim}\mu$ and define Then: In particular, by the law of large numbers $\widehat{\mathcal{E}}_n(g)\t

Figures (5)

  • Figure 1: Quadratic-over-linear benchmark. Left: Sample points from a 2D standard half-normal distribution and their images under the gradient map $\nabla f$ in \ref{['gql']}. Right:$L^2$ errors and training times for various dimensions.
  • Figure 2: Negative Logarithm: Numerical estimate of the Legendre transform computed with and without inverse sampling compared to the true Legendre transform in two dimensions.
  • Figure B1: Comparison of DGM and Time-DLT across three different moments of time $t=0.1,0.5, 1$, $d = 2.$
  • Figure B2: Distributions $\mu$ and $g^{-1}(\nu)$
  • Figure B3: Learning mixture of Gaussian distribution with the inverse sampling: leftmost -- train step 1, rightmost -- train step 3300.

Theorems & Definitions (2)

  • Lemma 1: Unbiased a posteriori error estimator
  • proof