On the Fenchel Duality between Strong Convexity and Lipschitz Continuous Gradient

TL;DR

The paper proves Fenchel duality between strong convexity and Lipschitz continuity of the gradient via a framework of equivalent convexity conditions that extend to non-differentiable functions. It establishes two key implications: μ-strong convexity of f implies a 1/μ-Lipschitz gradient for its Fenchel conjugate f*, and Lipschitz gradient of f with constant L implies μ = 1/L-strong convexity of f*. The approach systematizes equivalent first-order and subgradient conditions, demonstrates several non-equivalences that are nonetheless implied, and offers a simple, constructive proof that generalizes to related optimization properties.

Abstract

We provide a simple proof for the Fenchel duality between strong convexity and Lipschitz continuous gradient. To this end, we first establish equivalent conditions of convexity for a general function that may not be differentiable. By utilizing these equivalent conditions, we can directly obtain equivalent conditions for strong convexity and Lipschitz continuous gradient. Based on these results, we can easily prove Fenchel duality. Beside this main result, we also identify several conditions that are implied by strong convexity or Lipschitz continuous gradient, but are not necessarily equivalent to them. This means that these conditions are more general than strong convexity or Lipschitz continuous gradient themselves.

Theorems & Definitions (14)

• Theorem 1
• Lemma 1: Equivalence for Convexity
• proof
• Lemma 2: Equivalence for Strong Convexity
• proof
• Lemma 3: Implications of Strong Convexity (SC)
• proof
• Lemma 4
• proof : 1
• Lemma 5