Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Algorithmic Aspects of the Log-Laplace Transform and a Non-Euclidean Proximal Sampler

Sivakanth Gopi, Yin Tat Lee, Daogao Liu, Ruoqi Shen, Kevin Tian

TL;DR

This work develops a non-Euclidean proximal sampling framework by leveraging the log-Laplace transform (LLT) to regularize densities in general normed spaces. It establishes algorithmic LLT properties, including strong convexity–smoothness duality, self-concordance, isoperimetry, and TV bounds, to enable effective mixing of an alternating x–y sampler on an extended space. The authors prove a mixing-time bound for the proximal LLT sampler that generalizes Euclidean results and demonstrate practical gains in zeroth-order private convex optimization for ell_p and Schatten_p geometries, achieving competitive excess risk and improved value-query complexities under warm starts. They also discuss oracle-access strategies for LLT-based regularizers and outline open problems related to sharper mixing bounds, explicit-distribution samplers, and LLT extensions beyond proximal methods. Overall, the LLT provides a promising tool for designing efficient non-Euclidean samplers with potential impact on private optimization and beyond.

Abstract

The development of efficient sampling algorithms catering to non-Euclidean geometries has been a challenging endeavor, as discretization techniques which succeed in the Euclidean setting do not readily carry over to more general settings. We develop a non-Euclidean analog of the recent proximal sampler of [LST21], which naturally induces regularization by an object known as the log-Laplace transform (LLT) of a density. We prove new mathematical properties (with an algorithmic flavor) of the LLT, such as strong convexity-smoothness duality and an isoperimetric inequality, which are used to prove a mixing time on our proximal sampler matching [LST21] under a warm start. As our main application, we show our warm-started sampler improves the value oracle complexity of differentially private convex optimization in $\ell_p$ and Schatten-$p$ norms for $p \in [1, 2]$ to match the Euclidean setting [GLL22], while retaining state-of-the-art excess risk bounds [GLLST23]. We find our investigation of the LLT to be a promising proof-of-concept of its utility as a tool for designing samplers, and outline directions for future exploration.

Algorithmic Aspects of the Log-Laplace Transform and a Non-Euclidean Proximal Sampler

TL;DR

This work develops a non-Euclidean proximal sampling framework by leveraging the log-Laplace transform (LLT) to regularize densities in general normed spaces. It establishes algorithmic LLT properties, including strong convexity–smoothness duality, self-concordance, isoperimetry, and TV bounds, to enable effective mixing of an alternating x–y sampler on an extended space. The authors prove a mixing-time bound for the proximal LLT sampler that generalizes Euclidean results and demonstrate practical gains in zeroth-order private convex optimization for ell_p and Schatten_p geometries, achieving competitive excess risk and improved value-query complexities under warm starts. They also discuss oracle-access strategies for LLT-based regularizers and outline open problems related to sharper mixing bounds, explicit-distribution samplers, and LLT extensions beyond proximal methods. Overall, the LLT provides a promising tool for designing efficient non-Euclidean samplers with potential impact on private optimization and beyond.

Abstract

The development of efficient sampling algorithms catering to non-Euclidean geometries has been a challenging endeavor, as discretization techniques which succeed in the Euclidean setting do not readily carry over to more general settings. We develop a non-Euclidean analog of the recent proximal sampler of [LST21], which naturally induces regularization by an object known as the log-Laplace transform (LLT) of a density. We prove new mathematical properties (with an algorithmic flavor) of the LLT, such as strong convexity-smoothness duality and an isoperimetric inequality, which are used to prove a mixing time on our proximal sampler matching [LST21] under a warm start. As our main application, we show our warm-started sampler improves the value oracle complexity of differentially private convex optimization in and Schatten- norms for to match the Euclidean setting [GLL22], while retaining state-of-the-art excess risk bounds [GLLST23]. We find our investigation of the LLT to be a promising proof-of-concept of its utility as a tool for designing samplers, and outline directions for future exploration.
Paper Structure (49 sections, 37 theorems, 160 equations, 2 algorithms)

This paper contains 49 sections, 37 theorems, 160 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Non-Euclidean samplers.
  3. Proximal samplers.
  4. Our approach.
  5. Our results
  6. Algorithmic aspects of the LLT.
  7. Non-Euclidean proximal sampling.
  8. Zeroth-order private convex optimization.
  9. Our techniques
  10. Algorithmic aspects of the LLT.
  11. Non-Euclidean proximal sampling.
  12. Zeroth-order private convex optimization.
  13. Prior work
  14. Non-Euclidean sampling.
  15. Proximal sampling.
  16. ...and 34 more sections

Key Result

Lemma 1

Suppose $\phi: \mathbb{R}^d \to \mathbb{R}$ is convex and self-concordant. For $x,y \in \mathbb{R}^d$, if $d_\phi(x,y)\leq \delta - \delta^2 < 1$ for some $\delta \in (0, 1)$, then $\left\lVert y-x\right\rVert_{x}\leq \delta$.

Theorems & Definitions (63)

  • Lemma 1: nesterov2002riemannian, Lemma 3.1
  • Lemma 2: nemirovski2004interior, Section 2.2.1
  • Lemma 3: LLT derivatives
  • proof
  • Lemma 4: Self-concordance
  • proof
  • Lemma 5: Strong convexity-smoothness duality
  • proof
  • Lemma 6: Smoothness-strong convexity duality
  • proof
  • ...and 53 more