Trial factors for the look elsewhere effect in high energy physics

TL;DR

The paper tackles the look-elsewhere effect in high-energy resonance searches, where the signal location within a mass range is unknown and standard p-values are misleading. It employs Davies' upcrossing bound to bound the tail probability of the maximum likelihood ratio over the mass range, estimating the required upcrossings with a small Monte Carlo sample at a low threshold and relating them to the high-threshold tail. The method yields a practical p-value bound and a trial-factor interpretation, showing that the factor grows linearly with both the fixed-mass significance and the effective number of independent search regions, as demonstrated in a toy mass-bump model. It provides a fast, general framework for quantifying the LEE, extendable to multi-channel searches, and avoids extensive full-range MC simulations.

Abstract

When searching for a new resonance somewhere in a possible mass range, the significance of observing a local excess of events must take into account the probability of observing such an excess anywhere in the range. This is the so called "look elsewhere effect". The effect can be quantified in terms of a trial factor, which is the ratio between the probability of observing the excess at some fixed mass point, to the probability of observing it anywhere in the range. We propose a simple and fast procedure for estimating the trial factor, based on earlier results by Davies. We show that asymptotically, the trial factor grows linearly with the (fixed mass) significance.

Figures (5)

• Figure 1: (top) An example pseudo-experiment with background only. The solid line shows the best signal fit, while the dotted line shows the background fit. (bottom) The likelihood ratio test statistic $q(m)$. The dotted line marks the reference level $c_0$ with the upcrossings marked by the dark dots. Note the broadening of the fluctuations as $m$ increases, reflecting the increase in the signal gaussian width.
• Figure 2: (top) Distribution of $q(\hat{m})$. (bottom) Tail probability of $q(\hat{m})$. The solid line shows the result of the Monte Carlo simulation, the dotted red line is the predicted bound (eq. \ref{['eq3']}) with the estimated $\langle N(c_0) \rangle$ (see text). The yellow band represents the statistical uncertainty due to the limited sample size.
• Figure 3: The trial factor estimated from toy Monte Carlo simulations (solid line), with the upper bound of eq.(\ref{['eq3']}) (dotted black line) and the asymptotic approximation of eq.(\ref{['trial1']}) (dotted red line). The yellow band represents the statistical uncertainty due to the limited sample size.
• Figure 4: (top) Distribution of $q(\hat{m})$ for $s=2,3$. (bottom)Tail probability of $q(\hat{m})$. The solid lines shows the result of the Monte Carlo simulation, the dotted red lines are the predicted bound (eq. \ref{['eq3']}) with the estimated $\langle N(c_0) \rangle$ (see text).
• Figure 5: The trial factors estimated from toy Monte Carlo simulations (solid line), with the upper bound of eq.(\ref{['eq3']}) (dotted black line) and the asymptotic approximation of eq.(\ref{['trial2']}) (dotted red line). The yellow band represents the statistical uncertainty due to the limited sample size.