Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Collective intelligence in science: direct elicitation of diverse information from experts with unknown information structure

Alexey V. Osipov, Nikolay N. Osipov

TL;DR

The paper tackles the problem of aggregating deep, privately held information about a complex hypothesis $H$ when expert information structures are diverse and unknown. It proposes a self-resolving play-money prediction market entangled with a public chat, where all participants start with equal capital and trade as if the market resolves by $H$, while privately sharing information in an interpretable form. Through a rigorous epistemic-game-theoretic formalization, the authors show that, under plausible assumptions, the final market price equals the posterior $\pi(H\mid \Omega_{k_\infty})$ and that all beliefs align with the pooled information, enabling transparent information aggregation without requiring ground-truth validation. The work provides robustness analyses, relaxations of restrictive assumptions, and a complete formal framework to support large-scale collaborative scientific analyses, with potential to fund and organize complex open problems via tradable incentives.

Abstract

Suppose we need a deep collective analysis of an open scientific problem: there is a complex scientific hypothesis and a large online group of mutually unrelated experts with relevant private information of a diverse and unpredictable nature. This information may be results of experts' individual experiments, original reasoning of some of them, results of AI systems they use, etc. We propose a simple mechanism based on a self-resolving play-money prediction market entangled with a chat. We show that such a system can easily be brought to an equilibrium where participants directly share their private information on the hypothesis through the chat and trade as if the market were resolved in accordance with the truth of the hypothesis. This approach will lead to efficient aggregation of relevant information in a completely interpretable form even if the ground truth cannot be established and experts initially know nothing about each other and cannot perform complex Bayesian calculations. Finally, by rewarding the experts with some real assets proportionally to the play money they end up with, we can get an innovative way to fund large-scale collaborative studies of any type.

Collective intelligence in science: direct elicitation of diverse information from experts with unknown information structure

TL;DR

The paper tackles the problem of aggregating deep, privately held information about a complex hypothesis when expert information structures are diverse and unknown. It proposes a self-resolving play-money prediction market entangled with a public chat, where all participants start with equal capital and trade as if the market resolves by , while privately sharing information in an interpretable form. Through a rigorous epistemic-game-theoretic formalization, the authors show that, under plausible assumptions, the final market price equals the posterior and that all beliefs align with the pooled information, enabling transparent information aggregation without requiring ground-truth validation. The work provides robustness analyses, relaxations of restrictive assumptions, and a complete formal framework to support large-scale collaborative scientific analyses, with potential to fund and organize complex open problems via tradable incentives.

Abstract

Suppose we need a deep collective analysis of an open scientific problem: there is a complex scientific hypothesis and a large online group of mutually unrelated experts with relevant private information of a diverse and unpredictable nature. This information may be results of experts' individual experiments, original reasoning of some of them, results of AI systems they use, etc. We propose a simple mechanism based on a self-resolving play-money prediction market entangled with a chat. We show that such a system can easily be brought to an equilibrium where participants directly share their private information on the hypothesis through the chat and trade as if the market were resolved in accordance with the truth of the hypothesis. This approach will lead to efficient aggregation of relevant information in a completely interpretable form even if the ground truth cannot be established and experts initially know nothing about each other and cannot perform complex Bayesian calculations. Finally, by rewarding the experts with some real assets proportionally to the play money they end up with, we can get an innovative way to fund large-scale collaborative studies of any type.
Paper Structure (6 sections, 2 theorems, 37 equations)

This paper contains 6 sections, 2 theorems, 37 equations.

Table of Contents

  1. Problem
  2. Preliminaries
  3. Solution
  4. Robustness
  5. Formalization
  6. Acknowledgments

Key Result

Theorem 5.1

Suppose that whenever $n\in e_k$, $n$ believes the others in $e_k$ have the type $\langle\pi(H \mid \Omega_k)\rangle_k$. Then $\mathcal{M}$ is entangled with $H$.

Theorems & Definitions (8)

  • Definition 5.1
  • Definition 5.2
  • Definition 5.3
  • proof
  • Theorem 5.1: formalization of \ref{['it:T1']}
  • proof
  • Theorem 5.2: formalization of \ref{['it:T2']}
  • proof