Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A precise bare simulation approach to the minimization of some distances. II. Further Foundations

Michel Broniatowski, Wolfgang Stummer

TL;DR

Constrained optimization problems of a very huge amount of directed distances and generalized entropies -- and beyond -- can be tackled by a newly developed dimension-free extended bare simulation method, for obtaining both optima as well as optimizers.

Abstract

The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition. In our previous paper "A precise bare simulation approach to the minimization of some distances. I. Foundations", we obtained such kind of constrained optima by a new dimension-free precise bare (pure) simulation method, provided basically that (i) the underlying directed distance is of f-divergence type, and that (ii) this can be connected to a light-tailed probability distribution in a certain manner. In the present paper, we extend this approach such that constrained optimization problems of a very huge amount of directed distances and generalized entropies -- and beyond -- can be tackled by a newly developed dimension-free extended bare simulation method, for obtaining both optima as well as optimizers. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). For instance, we cover constrained optimizations of arbitrary f-divergences, Bregman distances, scaled Bregman distances and weighted Euclidean distances. The potential for wide-spread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences in various different research fields (which may also serve as an interdisciplinary interface).

A precise bare simulation approach to the minimization of some distances. II. Further Foundations

TL;DR

Constrained optimization problems of a very huge amount of directed distances and generalized entropies -- and beyond -- can be tackled by a newly developed dimension-free extended bare simulation method, for obtaining both optima as well as optimizers.

Abstract

The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition. In our previous paper "A precise bare simulation approach to the minimization of some distances. I. Foundations", we obtained such kind of constrained optima by a new dimension-free precise bare (pure) simulation method, provided basically that (i) the underlying directed distance is of f-divergence type, and that (ii) this can be connected to a light-tailed probability distribution in a certain manner. In the present paper, we extend this approach such that constrained optimization problems of a very huge amount of directed distances and generalized entropies -- and beyond -- can be tackled by a newly developed dimension-free extended bare simulation method, for obtaining both optima as well as optimizers. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). For instance, we cover constrained optimizations of arbitrary f-divergences, Bregman distances, scaled Bregman distances and weighted Euclidean distances. The potential for wide-spread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences in various different research fields (which may also serve as an interdisciplinary interface).
Paper Structure (51 sections, 51 theorems, 344 equations)

This paper contains 51 sections, 51 theorems, 344 equations.

Table of Contents

  1. Introduction
  2. A new minimization paradigm
  3. Deterministic Narrow-Sense Bare-Simulation-Optimization of $\varphi-$divergences
  4. Deterministic Narrow-Sense Bare-Simulation-Optimization of Bregman divergences
  5. Bare-Simulation-Method for General Deterministic Divergence-Optimization-Problems
  6. Further Divergences and Friends
  7. Minimization via Base-Divergence-Method 1
  8. Minimization via Base-Divergence-Method 2
  9. Maximization via Base-Divergence-Method 1
  10. Maximization via Base-Divergence-Method 2
  11. Deterministic Narrow-Sense Bare-Simulation-Optimization of $\varphi-$divergences with Constant-Component-Sum Side Constraint
  12. Deterministic Narrow-Sense Bare-Simulation-Optimization of Bregman Distances with Constant-Component-Sum Side Constraint
  13. Bare-Simulation-Method for General Deterministic Divergence-Optimization-Problems with Constant-Component-Sum Side Constraint
  14. Minimization via Base-Divergence-Method 1
  15. Maximization via Base-Divergence-Method 1
  16. ...and 36 more sections

Key Result

Theorem 9

Let $\mathbf{P} \in \mathbb{R}_{> 0}^{K}$, $M_{\mathbf{P}}:=\sum_{i=1}^{K}p_{i}>0$, and suppose that the divergence generator $\varphi$ satisfies Condition Condition Fi Tilda in Minimization above, with $\widetilde{\mathbb{\bbzeta}}$ (cf. brostu5:fo.link.var). Additionally, let $\widetilde{W}:=(\wi for any $\mathbf{\Omega }\subset \mathbb{R}^{K}$ with regularity properties regularity and finitene

Theorems & Definitions (101)

  • Definition 1
  • Remark 2
  • Definition 3
  • Remark 6
  • Remark 7
  • Remark 8
  • Theorem 9
  • Remark 10
  • Remark 11
  • Example 12
  • ...and 91 more