Finding an Upper Limit in the Presence of Unknown Background

TL;DR

The paper addresses setting upper limits on a signal when an unknown background cannot be reliably subtracted. It introduces two interval-based, unbinned methods—the Maximum Gap method and the Optimum Interval method—that yield true frequentist one-sided confidence intervals and are robust to unknown background distributions. By deriving the key statistic $C_0(x,\mu)$ for the maximum-gap case and extending to $C_n(x,\mu)$ and $C_{\mathrm{Max}}$ for the optimum-interval approach, the authors show these methods outperform conventional Poisson-based limits in low-count, background-uncertainty scenarios. The optimum interval method with $C_{\mathrm{Max}}$ provides the strongest, most reliable upper limits, is insensitive to binning and cut choices, and is suitable for rare-event searches such as WIMP experiments, with practical software support available.

Abstract

Experimenters report an upper limit if the signal they are trying to detect is non-existent or below their experiment's sensitivity. Such experiments may be contaminated with a background too poorly understood to subtract. If the background is distributed differently in some parameter from the expected signal, it is possible to take advantage of this difference to get a stronger limit than would be possible if the difference in distribution were ignored. We discuss the ``Maximum Gap'' method, which finds the best gap between events for setting an upper limit, and generalize to ``Optimum Interval'' methods, which use intervals with especially few events. These methods, which apply to the case of relatively small backgrounds, do not use binning, are relatively insensitive to cuts on the range of the parameter, are parameter independent (i.e., do not change when a one-one change of variables is made), and provide true, though possibly conservative, classical one-sided confidence intervals.

Figures (5)

• Figure 1: Illustration of the maximum gap method. The horizontal axis is some parameter, "$E$", measured for each event. The smooth curve is the signal expected for the proposed cross section, including any known background. The events from signal, known background, and unknown background are the small rectangles along the horizontal axis. The integral of the signal between two events is "$x_{\mathrm i}$".
• Figure 2: Plot of $\bar{C}_{\mathrm{Max}}(.9,\mu)$, the value of $C_{\mathrm{Max}}$ for which the 90% confidence level is reached, as a function of the total number of events $\mu$ expected in the experimental range.
• Figure 3: $\sigma_{\mathrm{Med}}/\sigma_{\mathrm{True}}$, the typical factor by which the upper limit cross section exceeds the true cross section, when $C_0$ is used (dotted lines), when $p_{\mathrm{Max}}$ is used (dash-dotted lines), when $C_{\mathrm{Max}}$ is used (dashed lines), and when the Poisson method is used (solid lines). These ratios are a function of $\mu$, the total number of events expected from the true cross section in the entire experimental range. For the upper figure (a) there is no background, and for the lower figure (b) there is just as much unknown background as there is signal, but the background is concentrated in a part of the experimental range that contains only half the total signal.
• Figure 4: Fraction of cases for test (b) (see text) in which the true cross section was higher than the upper limit on the cross section computed using $C_0$ (dotted), $p_{\mathrm{Max}}$ (dash-dotted) and $C_{\mathrm{Max}}$ (dashed).
• Figure 5: Plot of $\bar{p}_{\mathrm{Max}}(.9,\mu)$, the value of $p_{\mathrm{Max}}$ for which the 90% confidence level is reached, as a function of the total number of events $\mu$ expected in the experimental range.