Incentive-Theoretic Bayesian Inference for Collaborative Science

TL;DR

The paper addresses hypothesis testing when an agent with private beliefs chooses whether to run a trial and a principal must decide based on the outcome, proposing an incentive-theoretic Bayesian framework that leverages revealed preferences to infer about the agent's prior without the principal needing its own prior. It derives a posterior odds bound $\frac{P(\theta \in \Theta_1 \mid approve)}{P(\theta \in \Theta_0 \mid approve)} \ge \frac{C / R - \tau}{\tau}$, leading to a Bayes FDR bound $P(\theta \in \Theta_0 \mid approve) \le \tau R / C$ and guidance for selecting $\tau$ (e.g., $\tau \approx \alpha C / R$) to control false discoveries. The framework also yields a prior-free bound: if aggregate profits are nonnegative, the ratio of true to false positives is bounded below by $(C / R - \tau)/\tau$, independent of explicit priors. The practical implications include tailored p-value thresholds for clinical trials and insights for FDA-style regulation, showing how economic incentives shape inference quality and suggesting context-specific thresholds like $\tau \approx C /(4R)$ to cap Bayes FDR at 25%. Overall, the work bridges frequentist error control with Bayesian reasoning by using revealed information from strategic behavior, offering a sociotechnical lens for rigorous, incentive-aware statistical practice in collaborative science.

Abstract

Contemporary scientific research is a distributed, collaborative endeavor, carried out by teams of researchers, regulatory institutions, funding agencies, commercial partners, and scientific bodies, all interacting with each other and facing different incentives. To maintain scientific rigor, statistical methods should acknowledge this state of affairs. To this end, we study hypothesis testing when there is an agent (e.g., a researcher or a pharmaceutical company) with a private prior about an unknown parameter and a principal (e.g., a policymaker or regulator) who wishes to make decisions based on the parameter value. The agent chooses whether to run a statistical trial based on their private prior and then the result of the trial is used by the principal to reach a decision. We show how the principal can conduct statistical inference that leverages the information that is revealed by an agent's strategic behavior -- their choice to run a trial or not. In particular, we show how the principal can design a policy to elucidate partial information about the agent's private prior beliefs and use this to control the posterior probability of the null. One implication is a simple guideline for the choice of significance threshold in clinical trials: the type-I error level should be set to be strictly less than the cost of the trial divided by the firm's profit if the trial is successful.

Figures (1)

• Figure 1: Comparison of the fraction of false discoveries computed numerically to the upper bound from Theorem \ref{['thm:posterior_odds']} in the example of Section \ref{['sec:simple_example']}.

Theorems & Definitions (4)

• Theorem 1: Incentive-theoretic bound on the posterior odds of null
• Theorem 2: Prior-free incentive-theoretic bound
• proof : Proof of Theorem \ref{['thm:posterior_odds']}
• proof : Proof of Theorem \ref{['thm:betting']}