Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Confidence Level Computation for Combining Searches with Small Statistics

Thomas Junk

TL;DR

The paper presents an efficient, approximate method to compute confidence levels for excluding new particles when multiple small-statistics searches are combined. It uses a likelihood-ratio-based test statistic that factorizes across channels, with $CL_{s+b}$, $CL_b$, and $CL_s$ defined in the modified frequentist framework. By constructing and binning the PDF of the test statistic and convolving across channels, the approach avoids exhaustive enumeration and can incorporate uncorrelated systematic uncertainties via Gaussian averaging. The method is validated against exact calculations and Monte Carlo benchmarks, and is demonstrated on Higgs/MSSM LEP analyses and mock experiments, offering a practical tool for rapid model-scanning and multi-channel combinations.

Abstract

This article describes an efficient procedure for computing approximate confidence levels for searches for new particles where the expected signal and background levels are small enough to require the use of Poisson statistics. The results of many independent searches for the same particle may be combined easily, regardless of the discriminating variables which may be measured for the candidate events. The effects of systematic uncertainty in the signal and background models are incorporated in the confidence levels. The procedure described allows efficient computation of expected confidence levels.

Confidence Level Computation for Combining Searches with Small Statistics

TL;DR

The paper presents an efficient, approximate method to compute confidence levels for excluding new particles when multiple small-statistics searches are combined. It uses a likelihood-ratio-based test statistic that factorizes across channels, with , , and defined in the modified frequentist framework. By constructing and binning the PDF of the test statistic and convolving across channels, the approach avoids exhaustive enumeration and can incorporate uncorrelated systematic uncertainties via Gaussian averaging. The method is validated against exact calculations and Monte Carlo benchmarks, and is demonstrated on Higgs/MSSM LEP analyses and mock experiments, offering a practical tool for rapid model-scanning and multi-channel combinations.

Abstract

This article describes an efficient procedure for computing approximate confidence levels for searches for new particles where the expected signal and background levels are small enough to require the use of Poisson statistics. The results of many independent searches for the same particle may be combined easily, regardless of the discriminating variables which may be measured for the candidate events. The effects of systematic uncertainty in the signal and background models are incorporated in the confidence levels. The procedure described allows efficient computation of expected confidence levels.

Paper Structure

This paper contains 7 sections, 13 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Modified Frequentist Confidence Levels
  3. Confidence Level Calculation
  4. Systematic Uncertainty on Signal and Background
  5. Numerical Examples
  6. Limitations
  7. Summary

Figures (2)

  • Figure 1: The 95% CL upper bound on the number of events as a function of a hypothetical Higgs mass, using two test statistics, the likelihood ratio (filled circles) and events weighted by $s_i/b_i$ (empty circles). Candidates are shown with their respective mass resolutions at the bottom of the figure. The total background is four events expected to be uniformly distributed from zero to 100 GeV/$c^2$.
  • Figure 2: The false exclusion rate for the mock Higgs search experiment in the presence of a real signal at $m_{\mathrm H}$=77 GeV/$c^2$, for 95% CL computation. The error bars are hidden within the plot symbols. If a pure frequentist approach were taken (using $CL_{s+b}$), then the false exclusion probability would be flat at 5%.