Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Existence of Firth's modified estimates in binomial regression models

Mitsunori Ogawa, Yui Tomo

Abstract

In logistic regression modeling, Firth's modified estimator is widely used to address the issue of data separation, which results in the nonexistence of the maximum likelihood estimate. Firth's modified estimator can be formulated as a penalized maximum likelihood estimator in which Jeffreys' prior is adopted as the penalty term. Despite its widespread use in practice, the formal verification of the corresponding estimate's existence has not been established. In this study, we establish the existence theorem of Firth's modified estimate in binomial logistic regression models, assuming only the full column rankness of the design matrix. We also discuss other binomial regression models obtained through alternating link functions and prove the existence of similar penalized maximum likelihood estimates for such models.

Existence of Firth's modified estimates in binomial regression models

Abstract

In logistic regression modeling, Firth's modified estimator is widely used to address the issue of data separation, which results in the nonexistence of the maximum likelihood estimate. Firth's modified estimator can be formulated as a penalized maximum likelihood estimator in which Jeffreys' prior is adopted as the penalty term. Despite its widespread use in practice, the formal verification of the corresponding estimate's existence has not been established. In this study, we establish the existence theorem of Firth's modified estimate in binomial logistic regression models, assuming only the full column rankness of the design matrix. We also discuss other binomial regression models obtained through alternating link functions and prove the existence of similar penalized maximum likelihood estimates for such models.
Paper Structure (4 sections, 5 theorems, 17 equations)

This paper contains 4 sections, 5 theorems, 17 equations.

Table of Contents

  1. Introduction
  2. Existence guarantee in the logistic regression case
  3. Binomial regression models specified with other link functions
  4. Lemmas used in the proof of Proposition \ref{['prop:probit-cloglog']}

Key Result

Theorem 1

If $X$ is of full column rank, there exists a maximizer $\hat{\beta}\in{\mathbb R}^p$ of $l^*$ and the set of maximizers $\operatorname{argmax}_{\beta\in{\mathbb R}^p} l^*(\beta)$ of $l^*$ is bounded.

Theorems & Definitions (10)

  • Theorem 1
  • proof : Proof of Theorem \ref{['the:existence_b']}
  • Lemma 2
  • proof
  • Proposition 3
  • proof
  • Lemma A.1
  • proof
  • Lemma A.2
  • proof