Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Axiomatic Analysis of Distributionally Robust Optimization with $q$-Norm Ambiguity Sets for Probability Smoothing

Yoichi Izunaga, Kota Kurihara, Hokuto Nagano, Daiki Uchida

TL;DR

The paper tackles zero-frequency smoothing by introducing q-DRO, a distributionally robust optimization framework with a q-norm ambiguity set around the empirical distribution. It proves that the resulting estimator satisfies Positivity and Symmetry for all q and, for q in (1, ∞), Order Preservation under mild non-degeneracy, while showing the problem is a convex conic program and equivalent to regularized empirical loss minimization. Boundary analyses reveal that Order Preservation can fail for q = 1 and q = ∞, with numerical experiments confirming axiom verification for q = 2 and illustrating the regularization effect as the robustness radius grows. Overall, the study positions q-DRO as a principled, robust, and computationally tractable approach to probability smoothing with clear connections to convex optimization and regularization theory.

Abstract

We analyze the axiomatic properties of a class of probability estimators derived from Distributionally Robust Optimization (DRO) with $q$-norm ambiguity sets ($q$-DRO), a principled approach to the zero-frequency problem. While classical estimators such as Laplace smoothing are characterized by strong linearity axioms like Ratio Preservation, we show that $q$-DRO provides a flexible alternative that satisfies other desirable properties. We first prove that for any $q \in [1, \infty]$, the $q$-DRO estimator satisfies the fundamental axioms of Positivity and Symmetry. For the case of $q \in (1, \infty)$, we then prove that it also satisfies Order Preservation. Our analysis of the optimality conditions also reveals that the $q$-DRO formulation is equivalent to the regularized empirical loss minimization.

An Axiomatic Analysis of Distributionally Robust Optimization with $q$-Norm Ambiguity Sets for Probability Smoothing

TL;DR

The paper tackles zero-frequency smoothing by introducing q-DRO, a distributionally robust optimization framework with a q-norm ambiguity set around the empirical distribution. It proves that the resulting estimator satisfies Positivity and Symmetry for all q and, for q in (1, ∞), Order Preservation under mild non-degeneracy, while showing the problem is a convex conic program and equivalent to regularized empirical loss minimization. Boundary analyses reveal that Order Preservation can fail for q = 1 and q = ∞, with numerical experiments confirming axiom verification for q = 2 and illustrating the regularization effect as the robustness radius grows. Overall, the study positions q-DRO as a principled, robust, and computationally tractable approach to probability smoothing with clear connections to convex optimization and regularization theory.

Abstract

We analyze the axiomatic properties of a class of probability estimators derived from Distributionally Robust Optimization (DRO) with -norm ambiguity sets (-DRO), a principled approach to the zero-frequency problem. While classical estimators such as Laplace smoothing are characterized by strong linearity axioms like Ratio Preservation, we show that -DRO provides a flexible alternative that satisfies other desirable properties. We first prove that for any , the -DRO estimator satisfies the fundamental axioms of Positivity and Symmetry. For the case of , we then prove that it also satisfies Order Preservation. Our analysis of the optimality conditions also reveals that the -DRO formulation is equivalent to the regularized empirical loss minimization.

Paper Structure

This paper contains 19 sections, 3 theorems, 29 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Preliminaries: From Axiomatic Smoothing to a Distributionally Robust Formulation
  3. The Axiomatic Approach to Probability Smoothing
  4. A Principled Alternative: Distributionally Robust Optimization (DRO)
  5. Our DRO Formulation for Probability Smoothing
  6. Convex Reformulation of $q$-DRO
  7. Main Results: Axiomatic Properties of the $q$-DRO Estimator
  8. Positivity and Symmetry
  9. Order Preservation
  10. Discussion
  11. Validity of Assumption
  12. Equivalence to Regularized Empirical Loss Minimization
  13. Analysis of Boundary Cases: $q=1$ and $q=\infty$
  14. Case 1: $q=1$ ($q^{*}=\infty$)
  15. Case 2: $q=\infty$ ($q^{*}=1$)
  16. ...and 4 more sections

Key Result

Theorem 1

For any $q \in [1,\infty]$, the $q$-DRO estimator $\boldsymbol{x}$ satisfies Positivity and Symmetry.

Figures (2)

  • Figure 1: Comparison between empirical distribution $\hat{\boldsymbol{p}}$ and $2$-DRO estimator $\boldsymbol{x}$.
  • Figure 2: Sensitivity of the $2$-DRO estimator $\boldsymbol{x}$ to the robustness radius $\varepsilon$.

Theorems & Definitions (6)

  • Theorem 1
  • proof
  • Lemma 1
  • proof
  • Theorem 2
  • proof