Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exact Minimum-Volume Confidence Set Intersection for Multinomial Outcomes

Heguang Lin, Binhao Chen, Mengze Li, Daniel Pimentel-Alarcón, Matthew L. Malloy

TL;DR

This work addresses exactly how to certify whether minimum-volume confidence sets (MVCs) for multinomial outcomes intersect, a central task for robust A/B testing. It introduces a certified, tolerance-aware intersection test that avoids direct MVC geometry by exploiting likelihood ordering, which induces halfspace constraints in log-odds coordinates and enables adaptive triangle (or simplex) partitioning with computable p-value bounds. The approach delivers a sound yes/no decision (or an uncertain result within a prescribed margin) for k=3 and provides a principled extension to general k, including handling zero-count categories via face decomposition. The practical impact is tighter, reliable inference for discrete categorical data in small-sample regimes, with clear pathways to scalable implementations in real-world testing and reinforcement learning contexts.

Abstract

Computation of confidence sets is central to data science and machine learning, serving as the workhorse of A/B testing and underpinning the operation and analysis of reinforcement learning algorithms. Among all valid confidence sets for the multinomial parameter, minimum-volume confidence sets (MVCs) are optimal in that they minimize average volume, but they are defined as level sets of an exact p-value that is discontinuous and difficult to compute. Rather than attempting to characterize the geometry of MVCs directly, this paper studies a practically motivated decision problem: given two observed multinomial outcomes, can one certify whether their MVCs intersect? We present a certified, tolerance-aware algorithm for this intersection problem. The method exploits the fact that likelihood ordering induces halfspace constraints in log-odds coordinates, enabling adaptive geometric partitioning of parameter space and computable lower and upper bounds on p-values over each cell. For three categories, this yields an efficient and provably sound algorithm that either certifies intersection, certifies disjointness, or returns an indeterminate result when the decision lies within a prescribed margin. We further show how the approach extends to higher dimensions. The results demonstrate that, despite their irregular geometry, MVCs admit reliable certified decision procedures for core tasks in A/B testing.

Exact Minimum-Volume Confidence Set Intersection for Multinomial Outcomes

TL;DR

This work addresses exactly how to certify whether minimum-volume confidence sets (MVCs) for multinomial outcomes intersect, a central task for robust A/B testing. It introduces a certified, tolerance-aware intersection test that avoids direct MVC geometry by exploiting likelihood ordering, which induces halfspace constraints in log-odds coordinates and enables adaptive triangle (or simplex) partitioning with computable p-value bounds. The approach delivers a sound yes/no decision (or an uncertain result within a prescribed margin) for k=3 and provides a principled extension to general k, including handling zero-count categories via face decomposition. The practical impact is tighter, reliable inference for discrete categorical data in small-sample regimes, with clear pathways to scalable implementations in real-world testing and reinforcement learning contexts.

Abstract

Computation of confidence sets is central to data science and machine learning, serving as the workhorse of A/B testing and underpinning the operation and analysis of reinforcement learning algorithms. Among all valid confidence sets for the multinomial parameter, minimum-volume confidence sets (MVCs) are optimal in that they minimize average volume, but they are defined as level sets of an exact p-value that is discontinuous and difficult to compute. Rather than attempting to characterize the geometry of MVCs directly, this paper studies a practically motivated decision problem: given two observed multinomial outcomes, can one certify whether their MVCs intersect? We present a certified, tolerance-aware algorithm for this intersection problem. The method exploits the fact that likelihood ordering induces halfspace constraints in log-odds coordinates, enabling adaptive geometric partitioning of parameter space and computable lower and upper bounds on p-values over each cell. For three categories, this yields an efficient and provably sound algorithm that either certifies intersection, certifies disjointness, or returns an indeterminate result when the decision lies within a prescribed margin. We further show how the approach extends to higher dimensions. The results demonstrate that, despite their irregular geometry, MVCs admit reliable certified decision procedures for core tasks in A/B testing.
Paper Structure (26 sections, 7 theorems, 117 equations, 1 figure, 2 algorithms)

This paper contains 26 sections, 7 theorems, 117 equations, 1 figure, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notation and Basic Definitions
  3. AB-Testing Intersection Problem
  4. Certified Intersection Test for $k=3$
  5. Log-odds coordinates and outcome probabilities
  6. Likelihood-ordering halfspaces
  7. A compact search domain
  8. Cellwise lower bounds on multinomial probabilities
  9. Certified p-value bounds on a triangle
  10. Triangle refinement and the certified intersection test
  11. Correctness guarantee
  12. Computational complexity
  13. Extension to general $k$
  14. Log-odds coordinates in dimension $k-1$
  15. Halfspace structure and simplex cells
  16. ...and 11 more sections

Key Result

Proposition 1

(Minimum volume confidence set (MVCs) malloy2021ISIT). The MVCs are defined as and satisfy for any confidence set $\mathcal{C}_{\alpha}( \cdot )$; here $\mathrm{vol}( \cdot )$ denotes the Lebesgue measure. A proof can be found in malloy2021ISIT.

Figures (1)

  • Figure 1: Comparison of asymptotic and exact confidence sets on the simplex $\Delta_3$. The chi-square approximation (left) yields non-intersecting regions, whereas the MVCs (right) exhibit an intersection. Setting: $n=8, k =3$ and $\alpha=0.17$, two observed outcomes are $A = [1 , 6,1]$ and $B = [2,1,5].$

Theorems & Definitions (19)

  • Definition 1
  • Definition 2
  • Proposition 1
  • Definition 3: Robust intersection decision
  • Lemma 1: Halfspace form in $(u,v)$
  • proof
  • Lemma 2: A universal upper bound on the p-value
  • proof
  • Lemma 3: Vertex minimum for concave log-probabilities
  • proof
  • ...and 9 more