Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quasiprobabilistic Density Ratio Estimation with a Reverse Engineered Classification Loss Function

Matthew Drnevich, Stephen Jiggins, Kyle Cranmer

TL;DR

This work addresses density-ratio estimation when densities may be negative (quasiprobabilities) by generalizing the classifier-based ratio trick with a convex, reverse-engineered loss called REVERT, which maps classifier outputs to the real-valued density ratio via $r^*(\mathbf{x}) = T(s^*(\mathbf{x}))$. The authors derive the loss from the binary-classification Lagrangian and show that, for sigmoid outputs, $T(s)=\frac{1}{s}+\frac{1}{s-1}$ yields $r^*(\mathbf{x})$ in the real line, enabling quasiprobabilistic estimation. They validate the approach on a Standard Model vs SMEFT di-Higgs process at the LHC, comparing an MLP and Ratio of Signed Mixtures Models (RoSMM) trained with REVERT, and quantify performance with an extended Sliced-Wasserstein distance that handles signed measures. Across measures, REVERT-based methods achieve state-of-the-art results, highlighting the method's potential for simulation-based inference in physics and other domains where densities can be negative.

Abstract

We consider a generalization of the classifier-based density-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. The problem with most loss functions used for this task is that they implicitly define a relationship between the optimal classifier and the target quasiprobabilistic density ratio which is discontinuous or not surjective. We address these problems by introducing a convex loss function that is well-suited for both probabilistic and quasiprobabilistic density ratio estimation. To quantify performance, an extended version of the Sliced-Wasserstein distance is introduced which is compatible with quasiprobability distributions. We demonstrate our approach on a real-world example from particle physics, of di-Higgs production in association with jets via gluon-gluon fusion, and achieve state-of-the-art results.

Quasiprobabilistic Density Ratio Estimation with a Reverse Engineered Classification Loss Function

TL;DR

This work addresses density-ratio estimation when densities may be negative (quasiprobabilities) by generalizing the classifier-based ratio trick with a convex, reverse-engineered loss called REVERT, which maps classifier outputs to the real-valued density ratio via . The authors derive the loss from the binary-classification Lagrangian and show that, for sigmoid outputs, yields in the real line, enabling quasiprobabilistic estimation. They validate the approach on a Standard Model vs SMEFT di-Higgs process at the LHC, comparing an MLP and Ratio of Signed Mixtures Models (RoSMM) trained with REVERT, and quantify performance with an extended Sliced-Wasserstein distance that handles signed measures. Across measures, REVERT-based methods achieve state-of-the-art results, highlighting the method's potential for simulation-based inference in physics and other domains where densities can be negative.

Abstract

We consider a generalization of the classifier-based density-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. The problem with most loss functions used for this task is that they implicitly define a relationship between the optimal classifier and the target quasiprobabilistic density ratio which is discontinuous or not surjective. We address these problems by introducing a convex loss function that is well-suited for both probabilistic and quasiprobabilistic density ratio estimation. To quantify performance, an extended version of the Sliced-Wasserstein distance is introduced which is compatible with quasiprobability distributions. We demonstrate our approach on a real-world example from particle physics, of di-Higgs production in association with jets via gluon-gluon fusion, and achieve state-of-the-art results.
Paper Structure (17 sections, 22 equations, 2 figures, 6 tables)

This paper contains 17 sections, 22 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Density Ratio Estimation
  3. Deriving a Loss Function
  4. Reverse Engineered Ratio Trick Loss
  5. Application to Particle Physics: Standard Model Effective Field Theory in High Energy Physics
  6. Data Generation
  7. Machine Learning Setup
  8. Performance Measures
  9. Extended Sliced-Wasserstein Distance
  10. Results
  11. Conclusions
  12. Notation
  13. Risk Extrema
  14. Convexity of the Loss
  15. Additional Loss Functions
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Examples of transformations between the output space of a classifier $s$ which outputs values in the interval $(0,1)$ and the transformed output of the classifier using "ratio trick" transformations. $T_{\rm{BCE}}$ is the transformation associated with the binary cross-entropy loss function and $T_q$ is an example of a transformation suitable for quasiprobabilistic settings. $T_{q}$ can transform the classifier outputs to any real number, while $T_{\rm{BCE}}$ transforms all classifier outputs to positive numbers only.
  • Figure 2: The reweighting closure plots for the di-Higgs invariant mass ($m_{hh}$) where the reference (Standard Model) distribution is mapped to the target (SMEFT) distribution using the different density ratio estimation models.

Theorems & Definitions (1)

  • Definition 3.1