Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rank-based concordance for zero-inflated data: New representations, estimators, and sharp bounds

Jasper Arends, Guanjie Lyu, Mhamed Mesfioui, Elisa Perrone, Julien Trufin

TL;DR

This work addresses the challenge of measuring concordance for zero-inflated continuous data by reformulating Gini's gamma and Spearman's footrule through $Q(J,M)$ and $Q(J,W)$, and by correcting Spearman's $\rho$ for zero-inflation. It derives explicit representations and estimators for these measures, together with sharp, attainable bounds expressed in terms of zero-mass probabilities $p_1$ and $p_2$, and demonstrates practical estimation via simulations and two real-data case studies. The results show how zero inflation biases standard concordance measures and how the adjusted zero-inflated versions provide interpretable, bounded assessments of dependence. Overall, the paper provides a theoretical and applied framework for robust dependence analysis in zero-inflated contexts, with potential extensions to zero-inflated discrete settings and deeper connections between Kendall's tau and Spearman's rho in zero-inflated data.

Abstract

Quantifying concordance between two random variables is crucial in applications. Traditional estimation techniques for commonly used concordance measures, such as Gini's gamma or Spearman's rho, often fail when data contain ties. This is particularly problematic for zero-inflated data, characterized by a combination of discrete mass in zero and a continuous component, which frequently appear in insurance, weather forecasting, and biomedical applications. This study provides a new formulation of Gini's gamma and Spearman's footrule, two rank-based concordance measures that incorporate absolute rank differences, tailored to zero-inflated continuous distributions. Along the way, we correct an expression of Spearman's rho for zero-inflated data previously presented in the literature. The best-possible upper and lower bounds for these measures in zero-inflated continuous settings are established, making the estimators useful and interpretable in practice. We pair our theoretical results with simulations and two real-life applications in insurance and weather forecasting, respectively. Our results illustrate the impact of zero inflation on dependence estimation, emphasizing the benefits of appropriately adjusted zero-inflated measures.

Rank-based concordance for zero-inflated data: New representations, estimators, and sharp bounds

TL;DR

This work addresses the challenge of measuring concordance for zero-inflated continuous data by reformulating Gini's gamma and Spearman's footrule through and , and by correcting Spearman's for zero-inflation. It derives explicit representations and estimators for these measures, together with sharp, attainable bounds expressed in terms of zero-mass probabilities and , and demonstrates practical estimation via simulations and two real-data case studies. The results show how zero inflation biases standard concordance measures and how the adjusted zero-inflated versions provide interpretable, bounded assessments of dependence. Overall, the paper provides a theoretical and applied framework for robust dependence analysis in zero-inflated contexts, with potential extensions to zero-inflated discrete settings and deeper connections between Kendall's tau and Spearman's rho in zero-inflated data.

Abstract

Quantifying concordance between two random variables is crucial in applications. Traditional estimation techniques for commonly used concordance measures, such as Gini's gamma or Spearman's rho, often fail when data contain ties. This is particularly problematic for zero-inflated data, characterized by a combination of discrete mass in zero and a continuous component, which frequently appear in insurance, weather forecasting, and biomedical applications. This study provides a new formulation of Gini's gamma and Spearman's footrule, two rank-based concordance measures that incorporate absolute rank differences, tailored to zero-inflated continuous distributions. Along the way, we correct an expression of Spearman's rho for zero-inflated data previously presented in the literature. The best-possible upper and lower bounds for these measures in zero-inflated continuous settings are established, making the estimators useful and interpretable in practice. We pair our theoretical results with simulations and two real-life applications in insurance and weather forecasting, respectively. Our results illustrate the impact of zero inflation on dependence estimation, emphasizing the benefits of appropriately adjusted zero-inflated measures.
Paper Structure (18 sections, 5 theorems, 115 equations, 6 figures, 3 tables)

This paper contains 18 sections, 5 theorems, 115 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Results
  4. Gini's gamma and Spearman's footrule for zero-inflated continuous settings
  5. Correction to Spearman's rho for zero-inflated data
  6. Best-possible bounds
  7. Simulation study
  8. Case study
  9. Conclusions
  10. Proof of Theorems \ref{['thm:conc_upper']}, \ref{['thm:conc_lower']} and \ref{['thm:rho']}
  11. Proof of Theorem \ref{['thm:conc_upper']}
  12. Proof of Theorem \ref{['thm:conc_lower']}
  13. Proof of Theorem \ref{['thm:rho']}
  14. Proof of \ref{['thm:boundsgamma']}
  15. Proof of \ref{['thm:sper-rho-bounds']}
  16. ...and 3 more sections

Key Result

Theorem 3.1

Assuming $p_1 \leq p_2$, the concordance function relative to the upper Fréchet-Hoeffding copula is expressed as $C_M$ represents the concordance where both $X$ and $Y$ are positive and limited to $\rm (I)$, which is specified as

Figures (6)

  • Figure 1: Copulas (left) and corresponding sample support (right) for zero-inflated random variables $(X, Y)$ joined through the upper and lower Fréchet-Hoeffding copula bounds, (a) and (b) respectively, satisfying $p_1 \leq p_2$ and $p_1 + p_2 \leq 1$.
  • Figure 2: Numerical evaluation of the lower and upper bounds for Gini's gamma.
  • Figure 3: Numerical evaluation of the lower and upper bounds for Spearman's footrule.
  • Figure 4: Numerical evaluation of the lower and upper bounds for Spearman's rho.
  • Figure 5: Simulation results for the Fréchet copula with $\alpha = 0.5$ and $N = 150$, averaged over 1000 runs. The dashed lines mark the true values of the concordance measures.
  • ...and 1 more figures

Theorems & Definitions (10)

  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Theorem 3.3
  • proof
  • Theorem 3.4
  • proof
  • Theorem 3.5
  • proof