Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Marginal minimization and sup-norm expansions in perturbed optimization

Vladimir Spokoiny

TL;DR

This paper investigates marginal and partial optimization when the objective depends on a target variable $oldsymbol{x}$ and a nuisance variable $oldsymbol{s}$, introducing plug-in, alternating optimization (AO), and sup-norm estimation perspectives. It develops closed-form, local guarantees for semiparametric bias and linear perturbations using self-concordance-based smoothness and cross-derivative control, and extends to robust AO convergence results under block-strong convexity with contraction. The authors then establish sup-norm expansions for linearly and separably perturbed convex problems, providing tight, quadratic-in-perturbation bounds that support stable fixed-point updates in high-dimensional settings. Finally, they demonstrate these ideas on the Bradley–Terry–Luce model, where identifiability is addressed via penalization and the leading-term expansion closely matches the empirical behavior. The combination of marginal optimization analysis, AO convergence, and sup-norm perturbation theory yields practical, accurate tools for uncertainty quantification and robust estimation in perturbed optimization frameworks.

Abstract

Let the objective unction \( f \) depends on the target variable \( x \) along with a nuisance variable \( s \): \( f(v) = f(x,s) \). The goal is to identify the marginal solution \( x^{*} = \arg\min_{x} \min_{s} f(x,s) \). This paper discusses three related problems. The plugin approach widely used e.g. in inverse problems suggests to use a preliminary guess (pilot) \( \hat{s} \) and apply the solution of the partial optimization \( \hat{x} = \arg\min_{x} f(x,\hat{s}) \). The main question to address within this approach is the required quality of the pilot ensuring the prescribed accuracy of \( \hat{x} \). The popular \emph{alternating optimization} approach suggests the following procedure: given a starting guess \( x_{0} \), for \( t \geq 1 \), define \( s_{t} = \arg\min_{s} f(x_{t-1},s) \), and then \( x_{t} = \arg\min_{x} f(x,s_{t}) \). The main question here is the set of conditions ensuring a convergence of \( x_{t} \) to \( x^{*} \). Finally, the paper discusses an interesting connection between marginal optimization and sup-norm estimation. The basic idea is to consider one component of the variable \( v \) as a target and the rest as nuisance. In all cases, we provide accurate closed form results under realistic assumptions. The results are illustrated by one numerical example for the BTL model.

Marginal minimization and sup-norm expansions in perturbed optimization

TL;DR

This paper investigates marginal and partial optimization when the objective depends on a target variable and a nuisance variable , introducing plug-in, alternating optimization (AO), and sup-norm estimation perspectives. It develops closed-form, local guarantees for semiparametric bias and linear perturbations using self-concordance-based smoothness and cross-derivative control, and extends to robust AO convergence results under block-strong convexity with contraction. The authors then establish sup-norm expansions for linearly and separably perturbed convex problems, providing tight, quadratic-in-perturbation bounds that support stable fixed-point updates in high-dimensional settings. Finally, they demonstrate these ideas on the Bradley–Terry–Luce model, where identifiability is addressed via penalization and the leading-term expansion closely matches the empirical behavior. The combination of marginal optimization analysis, AO convergence, and sup-norm perturbation theory yields practical, accurate tools for uncertainty quantification and robust estimation in perturbed optimization frameworks.

Abstract

Let the objective unction depends on the target variable along with a nuisance variable : \( f(v) = f(x,s) \). The goal is to identify the marginal solution \( x^{*} = \arg\min_{x} \min_{s} f(x,s) \). This paper discusses three related problems. The plugin approach widely used e.g. in inverse problems suggests to use a preliminary guess (pilot) and apply the solution of the partial optimization \( \hat{x} = \arg\min_{x} f(x,\hat{s}) \). The main question to address within this approach is the required quality of the pilot ensuring the prescribed accuracy of . The popular \emph{alternating optimization} approach suggests the following procedure: given a starting guess , for , define \( s_{t} = \arg\min_{s} f(x_{t-1},s) \), and then \( x_{t} = \arg\min_{x} f(x,s_{t}) \). The main question here is the set of conditions ensuring a convergence of to . Finally, the paper discusses an interesting connection between marginal optimization and sup-norm estimation. The basic idea is to consider one component of the variable as a target and the rest as nuisance. In all cases, we provide accurate closed form results under realistic assumptions. The results are illustrated by one numerical example for the BTL model.
Paper Structure (18 sections, 19 theorems, 1 equation, 3 figures)

This paper contains 18 sections, 19 theorems, 1 equation, 3 figures.

Table of Contents

  1. Introduction
  2. Partial and marginal optimization
  3. Partial optimization
  4. Conditional optimization under (semi)orthogonality
  5. Semiparametric bias. A linear approximation
  6. A linear perturbation
  7. Alternating optimization
  8. Sup-norm expansions for linearly perturbed optimization
  9. Conditions
  10. A sup-norm expansion for a linear perturbation
  11. Sup-norm expansions for a separable perturbation
  12. Estimation for the Bradley-Terry-Luce model
  13. Optimization after linear perturbation
  14. Smoothness conditions
  15. A basic lemma
  16. ...and 3 more sections

Key Result

Proposition 2.1

Let $f_{\boldsymbol{s}}(\boldsymbol{x})$ be a strongly convex function with $f_{\boldsymbol{s}}(\boldsymbol{x}_{\boldsymbol{s}}) = \max_{\boldsymbol{x}} f_{\boldsymbol{s}}(\boldsymbol{x})$ and $\mathbb{F}_{\boldsymbol{s}} = \nabla^{2} f_{\boldsymbol{s}}(\boldsymbol{x}_{\boldsymbol{s}})$. Let $\bolds Then $\| \mathbb{D}_{\boldsymbol{s}} (\boldsymbol{x}_{\boldsymbol{s}} - \boldsymbol{x}^{*}) \| \leq

Figures (3)

  • Figure 5.1: Average $\rho_{1}$ value for different values of $n$.
  • Figure 5.2: Distribution of the leading term and the remainder for $n=100$.
  • Figure 5.3: Comparison of the leading term and the remainder for different $n$.

Theorems & Definitions (39)

  • Proposition 2.1
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • proof
  • Lemma 2.4
  • proof
  • Remark 2.1
  • Theorem 2.5
  • Remark 2.2
  • ...and 29 more