Algorithmic Bayesian Epistemology

TL;DR

This thesis establishes fundamental possibility and impossibility results about belief formation under a variety of restrictions, and lays the groundwork for further exploration of Bayesian epistemology.

Abstract

One aspect of the algorithmic lens in theoretical computer science is a view on other scientific disciplines that focuses on satisfactory solutions that adhere to real-world constraints, as opposed to solutions that would be optimal ignoring such constraints. The algorithmic lens has provided a unique and important perspective on many academic fields, including molecular biology, ecology, neuroscience, quantum physics, economics, and social science. This thesis applies the algorithmic lens to Bayesian epistemology. Traditional Bayesian epistemology provides a comprehensive framework for how an individual's beliefs should evolve upon receiving new information. However, these methods typically assume an exhaustive model of such information, including the correlation structure between different pieces of evidence. In reality, individuals might lack such an exhaustive model, while still needing to form beliefs. Beyond such informational constraints, an individual may be bounded by limited computation, or by limited communication with agents that have access to information, or by the strategic behavior of such agents. Even when these restrictions prevent the formation of a *perfectly* accurate belief, arriving at a *reasonably* accurate belief remains crucial. In this thesis, we establish fundamental possibility and impossibility results about belief formation under a variety of restrictions, and lay the groundwork for further exploration.

Figures (15)

• Figure 1: The thesis dependency structure. Solid arrows represent required background; dashed arrows represent recommended background.
• Figure 2: For the proper scoring rule derived from the function $G$ shown here, suppose that an expert reports a probability $x = 0.4$ on the "Yes" outcome. The expert will receive a score equal to the $y$-value of the red point on the right (roughly $0.3$) if the "Yes" outcome happens, and a score equal to the $y$-value of the red point on the left (roughly $0.7$) if the "No" outcome happens.
• Figure 3: The vertical distance shown is the Bregman divergence from $y$ to $x$ with respect to $G$, and is denoted $D_G(y \parallel x)$.
• Figure 4: For a random variable $Y$, let $B \subseteq A$ be sets of signals. Let $Y_A$ be the expected value of $Y$ conditioned on the signals in $A$, and define $Y_B$ analogously. In the space of random variables with inner product $\left\langle X_1, X_2\right\rangle := \mathbb{E}_{} \left[X_1 X_2\right]$, $Y_B$ is the orthogonal projection of $Y_A$ onto the subspace of random variables whose values only depend on the signals in $B$.
• Figure 5: Plots of $s_{\ell, \text{Opt}}$ for $\ell = 1, 2, 8, \infty$.

Theorems & Definitions (396)

• Definition 2.1.1
• Definition 2.1.2
• Definition 2.1.3
• Remark 2.1.4
• Definition 2.1.5
• Theorem 2.1.6: savage71
• Definition 2.1.6
• Proposition 2.1.7: banerjee2005clustering
• Example 2.2.1
• Example 2.2.2