Instance-Adaptive Hypothesis Tests with Heterogeneous Agents

TL;DR

This work addresses hypothesis testing in a setting with a heterogeneous population of strategic agents who hold private information about their priors. The authors propose menus of statistical contracts that induce agents to reveal their type and select type-optimal testing thresholds, thereby matching the performance of an oracle with full type knowledge. Central to the approach is a convex-function framework that links separating menus to strictly proper scoring rules, enabling incentive-compatible elicitation while quantifying information rents and screening costs. The results show that, under flexible contract design, the principal can achieve oracle-level error trade-offs with negligible financial cost, and they provide both constructive menus and insights into fixed-reward constraints. Numerical studies in Gaussian mean testing illustrate the geometry of separating menus and the improvements in FDR/TDR and Bayes risk achieved through instance-adaptive testing.

Abstract

We study hypothesis testing over a heterogeneous population of strategic agents with private information. Any single test applied uniformly across the population yields statistical error that is sub-optimal relative to the performance of an oracle given access to the private information. We show how it is possible to design menus of statistical contracts that pair type-optimal tests with payoff structures, inducing agents to self-select according to their private information. This separating menu elicits agent types and enables the principal to match the oracle performance even without a priori knowledge of the agent type. Our main result fully characterizes the collection of all separating menus that are instance-adaptive, matching oracle performance for an arbitrary population of heterogeneous agents. We identify designs where information elicitation is essentially costless, requiring negligible additional expense relative to a single-test benchmark, while improving statistical performance. Our work establishes a connection between proper scoring rules and menu design, showing how the structure of the hypothesis test constrains the elicitable information. Numerical examples illustrate the geometry of separating menus and the improvements they deliver in error trade-offs. Overall, our results connect statistical decision theory with mechanism design, demonstrating how heterogeneity and strategic participation can be harnessed to improve efficiency in hypothesis testing.

Figures (5)

• Figure 1: trade-offs between FDR and TDR for a population with two agent types: "good" and "bad". The light-blue region corresponds to all FDR-TDR pairs that can be achieved by any testing protocol. Its upper boundary (solid red) defines the Pareto frontier that can be achieved by an oracle that knows the agent type associated with each observation, and applies different test thresholds to each. The orange solid curve shows the result of applying a single uniform test to the full agent population. The dotted lines show the performance of applying a single test to only the "good" agents (purple line), and only the "bad" agents (green line); applying these two protocols also requires knowledge of the agent type.
• Figure 2: Principal's expected financial return \ref{['DefPrincipalReturn']} from offering a separating menu rather than the base contract $(\tau_{\bar{q}}, R_{\bar{q}}, C_{\bar{q}}) = (0.004, 100, 1.3)$ to each agent type. The separating menus are constructed using $\mathcal{G}$ from equation \ref{['eq:varyingR_G']}, where $\epsilon{(z)} = \eta\cdot(1-z)^2$ for the choices $\eta \in \{0.01, 0.1, 0.5, 1\}$.
• Figure 3: (a) Power functions $\beta_1(\tau)$ under alternative hypotheses $\theta_1 \in \{0.5, 1, 2\}$, with the maximum threshold $\tau$ satisfying condition \ref{['EqnPowerCondB']} marked. (b)--(d) Plots of the function $\mathcal{G}$ from equation \ref{['eq:constR_G']} under the same alternatives. The shaded area indicates the total information rent for uniformly distributed agent types under the separating menu constructed from $\mathcal{G}$.
• Figure 4: Plots of maximum utility \ref{['EqnOptInUtility']} achieved by each type $q \in [0,1]$ under a separating menu constructed for types $q \in \{ 0.3, 0.4, 0.5, 0.6, 0.7\}$. The dots mark the utilities of the designed types under truthful reporting, forming a discrete convex function $\mathcal{G}$. (a)-(b) Slack parameter $\epsilon{(t)} = 50$ and $\epsilon{(t)} = 100$, respectively, with $C_{t-1} = \frac{1}{2}[\ell_t + r_t]$. (c)-(d) Cost $C_{t-1} \approx \ell_t$ and $C_{t-1} \approx r_t$, respectively, with $\epsilon{(t)} = 50$.
• Figure 5: Plots of FDR gap \ref{['eq:fdr_gap_constR']} as a function of the reported type $p$, using power function $\tilde{\beta}_1(\tau)$ under misspecified alternative hypotheses $\theta_1 \neq 1$. (a) $\theta_1 \in \{1.1, 1.2, 1.3, 2\}$. (b) $\theta_1 \in \{0.6, 0.7, 0.8, 0.9\}$.

Theorems & Definitions (8)

• Theorem 1
• Corollary 1: Matching the oracle performance
• proof
• Corollary 2
• Corollary 3
• Proposition 1
• Lemma 1
• Lemma 2