Optimization of Scoring Rules

TL;DR

The paper develops a rigorous framework for optimizing scoring rules in principal–agent information elicitation, incorporating moral hazard and bounded-payoff constraints. It shows that the optimal mechanism to elicit the mean in one dimension is a simple V-shaped utility implemented via a two-bet betting scheme, while in multi-dimensional, especially symmetric settings, the optimal rule is a max-over-separate scheme that scores the most surprising dimension. For general (asymmetric) distributions, the max-over-separate rule remains approximately optimal (within constants), and only knowledge of the prior mean is needed to implement it; the approach extends to eliciting full distributions with analogous convex-utility characterizations. The results demonstrate that linking incentives across dimensions is crucial to sustain effort and improve information quality, often outperforming naive separable scoring rules and traditional quadratic scoring. Practically, the work provides design principles for exams, forecasting, and information procurement, and shows that in some cases eliciting full distribution information is significantly more incentivizing even when marginal means suffice for downstream decisions.

Abstract

We characterize the optimal reward functions (scoring rules) that incentivize an agent to acquire information and report it truthfully to the principal. The optimal scoring rules let the agent make a simple binary bet in single-dimensional problems, and choose the dimension with the most surprising signal to be scored on in symmetric multi-dimensional problems. This scoring rule format remains approximately optimal for asymmetric distributions. These results demonstrate the importance of linking incentives to obtain high-quality information in multi-dimensional problems. In contrast, standard scoring rules like the quadratic scoring rule, or averages of single-dimensional scoring rules can be far from optimal.

Figures (4)

• Figure 1: The figure on the left hand side illustrates the bounded constraint for proper scoring rule for single dimensional states. The figure on the right hand side characterizes the optimal scoring rule (solid line) for single dimensional states. In this figure, for any convex function $u$ (dotted line) that induces a bounded scoring rule, there exists another convex function $\tilde{u}$ (solid line) which also induces a bounded scoring rule and weakly improves the objective.
• Figure 2: This figure illustrates the scoring rule $S^*$ as a function of the realized state $\theta$. The dashed line is the score function when the report $r < \mu_{D}$, and the solid line is the score function when the report $r \geq \mu_{D}$.
• Figure 3: This figure depicts a two-dimensional state space. The state space $\Theta = [0,1]^2$ and its point reflection around the prior mean $\mu_{D}$ are shaded in gray. The extended report and state space are depicted by the region within the thick black rectangle.
• Figure 4: The figure on the left hand side illustrates a hyperplane for report $r'$ on the boundary of the report space, which is shifted from a tangent plane of $u$ at the boundary $r'$. The figure on the right hand side illustrates the extended utility function $\tilde{u}$ that takes the supremum over all hyperplanes shifted from the feasible tangent planes to intersect with the $(\mu_{D},0)$ point.

Theorems & Definitions (103)

• Definition 1: Proper
• Definition 2: Boundedness
• Definition 3
• Lemma 3.1: AF-12
• Lemma 3.2
• Lemma 4.1
• Definition 4
• Theorem 1
• Definition 5
• Claim 5.1