Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Double-Bayesian Learning

Stefan Jaeger

TL;DR

The proposed approach understands that Bayesian learning is tantamount to finding a base for a logarithmic function measuring uncertainty, with solutions being fixed points, and following this approach, the golden ratio describes possible solutions satisfying Bayes' theorem.

Abstract

Contemporary machine learning methods will try to approach the Bayes error, as it is the lowest possible error any model can achieve. This paper postulates that any decision is composed of not one but two Bayesian decisions and that decision-making is, therefore, a double-Bayesian process. The paper shows how this duality implies intrinsic uncertainty in decisions and how it incorporates explainability. The proposed approach understands that Bayesian learning is tantamount to finding a base for a logarithmic function measuring uncertainty, with solutions being fixed points. Furthermore, following this approach, the golden ratio describes possible solutions satisfying Bayes' theorem. The double-Bayesian framework suggests using a learning rate and momentum weight with values similar to those used in the literature to train neural networks with stochastic gradient descent.

Double-Bayesian Learning

TL;DR

The proposed approach understands that Bayesian learning is tantamount to finding a base for a logarithmic function measuring uncertainty, with solutions being fixed points, and following this approach, the golden ratio describes possible solutions satisfying Bayes' theorem.

Abstract

Contemporary machine learning methods will try to approach the Bayes error, as it is the lowest possible error any model can achieve. This paper postulates that any decision is composed of not one but two Bayesian decisions and that decision-making is, therefore, a double-Bayesian process. The paper shows how this duality implies intrinsic uncertainty in decisions and how it incorporates explainability. The proposed approach understands that Bayesian learning is tantamount to finding a base for a logarithmic function measuring uncertainty, with solutions being fixed points. Furthermore, following this approach, the golden ratio describes possible solutions satisfying Bayes' theorem. The double-Bayesian framework suggests using a learning rate and momentum weight with values similar to those used in the literature to train neural networks with stochastic gradient descent.

Paper Structure

This paper contains 14 sections, 24 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Dual decisions
  3. Bayes theorem
  4. Double-Bayesian framework
  5. Fixpoint solutions
  6. Golden ratio
  7. Theoretical implications
  8. Discussion
  9. Conclusion
  10. Appendix / supplemental material
  11. Experiments
  12. Results
  13. Computational environment and runtime
  14. Computing cluster

Figures (5)

  • Figure 1: An image of Rubin's vase (left) and its inverted counterpart (right) - rubin2015
  • Figure 2: A slightly enlarged example from the MNIST dataset showing a handwritten digit (4).
  • Figure 3: Grid search results for MNIST
  • Figure 4: Grid search results for MNIST using only 50% of the training data
  • Figure 5: GPU computing cluster