Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Quasiprobabilistic Likelihood Ratio Estimation with Negatively Weighted Data

Matthew Drnevich, Stephen Jiggins, Judith Katzy, Kyle Cranmer

TL;DR

This work considers a generalization of the likelihood-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative, and to importance sampling where the importance weights can be negative.

Abstract

Motivated by real-world situations found in high energy particle physics, we consider a generalisation of the likelihood-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. By extension, this framing also applies to importance sampling in a setting where the importance weights can be negative. The presence of negative densities and negative weights, pose an array of challenges to traditional neural likelihood ratio estimation methods. We address these challenges by introducing a novel loss function. In addition, we introduce a new model architecture based on the decomposition of a likelihood ratio using signed mixture models, providing a second strategy for overcoming these challenges. Finally, we demonstrate our approach on a pedagogical example and a real-world example from particle physics.

Neural Quasiprobabilistic Likelihood Ratio Estimation with Negatively Weighted Data

TL;DR

This work considers a generalization of the likelihood-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative, and to importance sampling where the importance weights can be negative.

Abstract

Motivated by real-world situations found in high energy particle physics, we consider a generalisation of the likelihood-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. By extension, this framing also applies to importance sampling in a setting where the importance weights can be negative. The presence of negative densities and negative weights, pose an array of challenges to traditional neural likelihood ratio estimation methods. We address these challenges by introducing a novel loss function. In addition, we introduce a new model architecture based on the decomposition of a likelihood ratio using signed mixture models, providing a second strategy for overcoming these challenges. Finally, we demonstrate our approach on a pedagogical example and a real-world example from particle physics.

Paper Structure

This paper contains 33 sections, 3 theorems, 91 equations, 29 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Importance Sampling and Negative Weights
  3. Negative Weights: Where do we find them?
  4. Related Work
  5. Loss Functions for Likelihood Ratio Estimation
  6. Ratio of Signed Mixtures Model
  7. Further Optimizing the Ratio of Signed Mixtures Model
  8. Pedagogical Example: Signed Gaussian Mixture Models
  9. Signed Gaussian Mixture
  10. Data Generation
  11. Machine Learning Setup
  12. Model Optimization and Negative Weights
  13. Results
  14. Application to Particle Physics: Standard Model Effective Field Theory in High Energy Physics
  15. Data Generation
  16. ...and 18 more sections

Key Result

Lemma 6.1

Let $f: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$ be any function that can be written in the form $f(\mathbf{x},\mathbf{y}) = \sum_j g_j(\mathbf{x})h_j(\mathbf{y})$ for some functions $g_j:\mathcal{X} \to \mathbb{R}$ and $h_j : \mathcal{Y} \to \mathbb{R}$ then

Figures (29)

  • Figure 1: Relationship between the likelihood ratio $r$ and the optimal classifier $s^{*}$, for the MSE and PARE loss functions, as given by equations \ref{['eq:inv_ratio_trick']} & \ref{['eq:pare_loss']}, respectively. The dashed line represents the range of $s^{*}$ and $r$, encompassed by the Signed LR dataset described in Section \ref{['sec:AppToyModel']}.
  • Figure 2: Signed Gaussian mixture reference distribution for the Nonnegative & Signed LR test case, using the settings in Table \ref{['tab:toy_settings']}. The one-dimensional distributions on the sides represent the marginal $X$ and $Y$ coordinate distributions.
  • Figure 3: Signed Gaussian mixture target distribution for the Nonnegative LR test case, using the settings in Table \ref{['tab:toy_settings']}. The one-dimensional distributions on the sides represent the marginal $X$ and $Y$ coordinate distributions.
  • Figure 4: Signed Gaussian mixture target distribution for the Signed LR test case, using the settings in Table \ref{['tab:toy_settings']}. The one-dimensional distributions on the sides represent the marginal $X$ and $Y$ coordinate distributions.
  • Figure 5: Reference and target radial distributions for the Nonnegative and Signed LR test cases, using the settings in Table \ref{['tab:toy_settings']}.
  • ...and 24 more figures

Theorems & Definitions (11)

  • Lemma 6.1
  • Proposition 6.2
  • Remark
  • proof
  • Proposition 6.3
  • proof
  • proof
  • proof
  • proof
  • proof
  • ...and 1 more