Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solomonoff induction

Tom F. Sterkenburg

Abstract

This chapter discusses the Solomonoff approach to universal prediction. The crucial ingredient in the approach is the notion of computability, and I present the main idea as an attempt to meet two plausible computability desiderata for a universal predictor. This attempt is unsuccessful, which is shown by a generalization of a diagonalization argument due to Putnam. I then critically discuss purported gains of the approach, in particular it providing a foundation for the methodological principle of Occam's razor, and it serving as a theoretical ideal for the development of machine learning methods.

Solomonoff induction

Abstract

This chapter discusses the Solomonoff approach to universal prediction. The crucial ingredient in the approach is the notion of computability, and I present the main idea as an attempt to meet two plausible computability desiderata for a universal predictor. This attempt is unsuccessful, which is shown by a generalization of a diagonalization argument due to Putnam. I then critically discuss purported gains of the approach, in particular it providing a foundation for the methodological principle of Occam's razor, and it serving as a theoretical ideal for the development of machine learning methods.
Paper Structure (33 sections, 4 theorems, 21 equations)

This paper contains 33 sections, 4 theorems, 21 equations.

Table of Contents

  1. Introduction
  2. Universal prediction
  3. Sequential prediction
  4. Universal reliability
  5. Reliability for computability
  6. A universal mixture
  7. Universal optimality
  8. Optimality among all methods
  9. Aggregating predictors
  10. Optimality
  11. The universal aggregator
  12. The diagonal argument
  13. The Solomonoff-Levin approach
  14. Semi-computable semi-measures
  15. Computable functions
  16. ...and 18 more sections

Key Result

Theorem 2

For mixture predictor $\textsf{p}^\mathcal{H}_w$, for all $\mu \in \mathcal{H}$, with $\mu$-probability 1, $\textsf{p}^\mathcal{H}_w(\pmb{x}^t,x_{t+1}) \xrightarrow{t \rightarrow \infty} \mu(x_{t+1}\mid \pmb{x}^t).$

Theorems & Definitions (10)

  • Definition 1: Bayesian mixture
  • Theorem 2: Bayesian consistency
  • proof
  • Theorem 3: Optimality
  • proof
  • Theorem 4: Putnam's diagonalization
  • proof
  • proof
  • Theorem 6
  • proof