Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unbiased Estimating Equation on Inverse Divergence and Its Conditions

Masahiro Kobayashi, Kazuho Watanabe

TL;DR

The paper addresses when the estimating equation for the loss $L(oldsymbol{ heta}) = rac{1}{n} ext{+++} n f(d_oldsymbol{phi}(oldsymbol{x}_i,oldsymbol{ heta}))$ under inverse divergence is unbiased without a bias-correction term. It introduces two model families, Inverse Gaussian Type (IGT) and Generalized IGT (GIGT) mixtures, and derives explicit integral conditions on $f$ and the generating function $g$ (e.g., $I(1)< ty$ and $ int_0^ty g(t)f'(t) dt<6$) that guarantee unbiasedness, with corresponding results for multi-dimensional data via the MIGT distribution and a double-integral criterion. The main contributions are the precise necessary-and-sufficient conditions for unbiasedness in 1D and the multi-dimensional extension, clarifying how the choice of $f$ and the generating function $g$ interacts with the underlying distribution class. The findings illuminate when latent biases can be effectively eliminated in robust estimation using inverse divergence, linking these estimators to (continuous) Bregman distributions and the regular exponential family. Practically, the work provides rigorous criteria for selecting $f$ and model families to achieve unbiased estimation without computationally intractable bias corrections, even under outliers.

Abstract

This paper focuses on the Bregman divergence defined by the reciprocal function, called the inverse divergence. For the loss function defined by the monotonically increasing function $f$ and inverse divergence, the conditions for the statistical model and function $f$ under which the estimating equation is unbiased are clarified. Specifically, we characterize two types of statistical models, an inverse Gaussian type and a mixture of generalized inverse Gaussian type distributions, to show that the conditions for the function $f$ are different for each model. We also define Bregman divergence as a linear sum over the dimensions of the inverse divergence and extend the results to the multi-dimensional case.

Unbiased Estimating Equation on Inverse Divergence and Its Conditions

TL;DR

The paper addresses when the estimating equation for the loss under inverse divergence is unbiased without a bias-correction term. It introduces two model families, Inverse Gaussian Type (IGT) and Generalized IGT (GIGT) mixtures, and derives explicit integral conditions on and the generating function (e.g., and ) that guarantee unbiasedness, with corresponding results for multi-dimensional data via the MIGT distribution and a double-integral criterion. The main contributions are the precise necessary-and-sufficient conditions for unbiasedness in 1D and the multi-dimensional extension, clarifying how the choice of and the generating function interacts with the underlying distribution class. The findings illuminate when latent biases can be effectively eliminated in robust estimation using inverse divergence, linking these estimators to (continuous) Bregman distributions and the regular exponential family. Practically, the work provides rigorous criteria for selecting and model families to achieve unbiased estimation without computationally intractable bias corrections, even under outliers.

Abstract

This paper focuses on the Bregman divergence defined by the reciprocal function, called the inverse divergence. For the loss function defined by the monotonically increasing function and inverse divergence, the conditions for the statistical model and function under which the estimating equation is unbiased are clarified. Specifically, we characterize two types of statistical models, an inverse Gaussian type and a mixture of generalized inverse Gaussian type distributions, to show that the conditions for the function are different for each model. We also define Bregman divergence as a linear sum over the dimensions of the inverse divergence and extend the results to the multi-dimensional case.
Paper Structure (14 sections, 7 theorems, 35 equations, 2 tables)

This paper contains 14 sections, 7 theorems, 35 equations, 2 tables.

Table of Contents

  1. Introduction
  2. $f$-separable Bregman distortion measures
  3. Problem setting and loss function
  4. Conditions for unbiased estimating equation
  5. Existing results
  6. Models and conditions under inverse divergence
  7. Inverse Gaussian type (IGT) distribution
  8. Generalized IGT (GIGT) mixture distribution
  9. Discussion
  10. Extension to multi-dimensional case
  11. Conclusion
  12. Detailed proof of Theorem \ref{['theorem:igt']}
  13. Proof of Theorem \ref{['theorem:multi_igt']}
  14. Lemmas

Key Result

Theorem 1

Under Assumption assp:igt, the estimating equation without a bias correction term equivalently, eq:bias0 holds if and only if holds for the combination of the function $f$ and the statistical model eq:igt.

Theorems & Definitions (12)

  • Definition 1: IGT distribution igt
  • Theorem 1
  • Proof 1
  • Corollary 1
  • Definition 2: GIGT distribution
  • Corollary 2
  • Definition 3: Multivariate IGT (MIGT) distribution
  • Theorem 2
  • Corollary 3
  • Proof 2
  • ...and 2 more