Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Resolving Prediction Markets for Unverifiable Outcomes

Siddarth Srinivasan, Ezra Karger, Yiling Chen

TL;DR

The paper introduces the first incentive-compatible self-resolving prediction market that aggregates information without direct ground truth. It achieves truthful reporting as a strict Perfect Bayesian Equilibrium by paying agents via negative cross-entropy against a reference agent, whose access to independent informational substitutes minimizes the influence of any single report. A sequential market with random termination and a fixed terminal reference agent provides a natural aggregation of beliefs while ensuring zero payoff in uninformative equilibria. The mechanism can be implemented with cross-entropy market scoring rules or via a cost-function-based automated market maker, and extensions to parallel markets and alternative termination schemes are discussed for practical deployment and empirical testing.

Abstract

Prediction markets elicit and aggregate beliefs by paying agents based on how close their predictions are to a verifiable future outcome. However, outcomes of many important questions are difficult to verify or unverifiable, in that the ground truth may be hard or impossible to access. We present a novel incentive-compatible prediction market mechanism to elicit and efficiently aggregate information from a pool of agents without observing the outcome, by paying agents the negative cross-entropy between their prediction and that of a carefully chosen reference agent. Our key insight is that a reference agent with access to more information can serve as a reasonable proxy for the ground truth. We use this insight to propose self-resolving prediction markets that terminate with some probability after every report and pay all but a few agents based on the final prediction. The final agent is chosen as the reference agent since they observe the full history of market forecasts, and thus have more information by design. We show that it is a perfect Bayesian equilibrium (PBE) for all agents to report truthfully in our mechanism and to believe that all other agents report truthfully. Although primarily of interest for unverifiable outcomes, this design is also applicable for verifiable outcomes.

Self-Resolving Prediction Markets for Unverifiable Outcomes

TL;DR

The paper introduces the first incentive-compatible self-resolving prediction market that aggregates information without direct ground truth. It achieves truthful reporting as a strict Perfect Bayesian Equilibrium by paying agents via negative cross-entropy against a reference agent, whose access to independent informational substitutes minimizes the influence of any single report. A sequential market with random termination and a fixed terminal reference agent provides a natural aggregation of beliefs while ensuring zero payoff in uninformative equilibria. The mechanism can be implemented with cross-entropy market scoring rules or via a cost-function-based automated market maker, and extensions to parallel markets and alternative termination schemes are discussed for practical deployment and empirical testing.

Abstract

Prediction markets elicit and aggregate beliefs by paying agents based on how close their predictions are to a verifiable future outcome. However, outcomes of many important questions are difficult to verify or unverifiable, in that the ground truth may be hard or impossible to access. We present a novel incentive-compatible prediction market mechanism to elicit and efficiently aggregate information from a pool of agents without observing the outcome, by paying agents the negative cross-entropy between their prediction and that of a carefully chosen reference agent. Our key insight is that a reference agent with access to more information can serve as a reasonable proxy for the ground truth. We use this insight to propose self-resolving prediction markets that terminate with some probability after every report and pay all but a few agents based on the final prediction. The final agent is chosen as the reference agent since they observe the full history of market forecasts, and thus have more information by design. We show that it is a perfect Bayesian equilibrium (PBE) for all agents to report truthfully in our mechanism and to believe that all other agents report truthfully. Although primarily of interest for unverifiable outcomes, this design is also applicable for verifiable outcomes.
Paper Structure (53 sections, 11 theorems, 61 equations, 8 figures)

This paper contains 53 sections, 11 theorems, 61 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Contributions
  3. Outline
  4. Related Work
  5. Model
  6. Peer Prediction
  7. Prediction Market
  8. Notation
  9. Assumptions
  10. Background
  11. Prediction Markets
  12. Informational Substitutes
  13. Perfect Bayesian Equilibrium
  14. Incentives under Cross-Entropy Scoring Rules
  15. Setup
  16. ...and 38 more sections

Key Result

Lemma 1

Agent $t$'s expectation of agent $r$'s true posterior ${\tilde{p}}_1^{(r)} = \mathbb{P}(Y=1|x^{(r)}, \tilde{x}^{(t)}, x^{(s)})$ is: where $\Delta(\Omega_{X_r}, \tilde{x}^{(t)}, {x}^{(t)}, x^{(s)}) = \mu(\Omega_{X_r}, \tilde{x}^{(t)}, x^{(s)}) \cdot \rho(\tilde{x}^{(t)}, x^{(t)}, x^{(s)})$, with

Figures (8)

  • Figure 1: Illustration of a self-resolving prediction market. Each node represents an agent reporting a prediction to the mechanism, and the mechanism terminates with probability $\alpha$ after each report. Payouts for the first $T-k$ agents are determined using a negative cross-entropy market scoring rule with respect to the terminal agent $T$, while the last $k$ agents receive a flat payout $R$. $k$ can be chosen to be large enough so the mechanism is strictly truthful.
  • Figure 2: Minimum number of informational substitutes $k_{min}$ that the reference agent needs to guarantee that the incentive to deviate from truthful reporting is no more than $\varepsilon$ as a function of the prior market prediction $y_1^{(t-1)}$, for different choices of $(\delta, \eta)$ when $\tau-$granularity is not known.
  • Figure 3: (Cross-Entropy Scoring Rules) Plot of the upper bound of gain in deviating from truthfulness as a function of the adjustment term $\Delta$. Observe that as the adjustment term $\Delta \to 0$, the upper bound $\mathcal{D}_\eta(\Delta, {\bf y}) \to 0$ as well, showing that the incentive to deviate can be made to disappear by sending the adjustment term to zero.
  • Figure 4: (Cross-Entropy Scoring Rules) Plot of maximum allowed adjustment $\varepsilon'$ to guarantee that the incentive to deviate from truthful reporting is no more than $\varepsilon$ as a function of the prior on $Y=1$ as $y_1$, for various choices of $\varepsilon$.
  • Figure 5: (Cross-Entropy Scoring Rules) Plot of minimum number of information substitutes that the reference agent needs to guarantee that the incentive to deviate from truthful reporting is no more than $\varepsilon$ as a function of the prior $y_1$ under cross-entropy scoring rules, for different choices of $(\delta, \eta)$.
  • ...and 3 more figures

Theorems & Definitions (31)

  • Definition 1: $(1-\delta^{(j)})$-Bhattacharyya Coefficient
  • Definition 2: $\tau-$granularity
  • Definition 3: Scoring Rules, Proper Scoring Rules, and Strictly Proper Scoring Rules
  • Definition 4: Log Scoring Rule
  • Definition 5: Cross-Entropy Scoring Rule
  • Definition 6: $S$-Market Scoring Rule
  • Definition 7: Cross-Entropy Market Scoring Rule (CE-MSR)
  • Lemma 1
  • Corollary 1: Martingale Property of Truthful Report
  • Theorem 1
  • ...and 21 more