Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Information Theoretic Proof of the Radon-Nikodym Theorem

Peter Harremoës

Abstract

The Radon-Nikodym theorem plays a significant role in the definition of Shannon entropy, f-divergences, and other basic quantities in information theory. The existence of Radon Nikodym derivates appear in many text books in measure theory but in text books on probability or information theory it is often omitted because the proof is often considered to be too difficult.

An Information Theoretic Proof of the Radon-Nikodym Theorem

Abstract

The Radon-Nikodym theorem plays a significant role in the definition of Shannon entropy, f-divergences, and other basic quantities in information theory. The existence of Radon Nikodym derivates appear in many text books in measure theory but in text books on probability or information theory it is often omitted because the proof is often considered to be too difficult.
Paper Structure (11 sections, 13 theorems, 42 equations)

This paper contains 11 sections, 13 theorems, 42 equations.

Table of Contents

  1. Introduction
  2. Notation and terminology
  3. Lattices and valuations
  4. Shannon inequalities on lattices of functional dependencies
  5. Concept lattices
  6. Valuations and distributive lattices
  7. Information divergences
  8. Pointwise convergence
  9. Almost sure pointwise convergence
  10. Finite divergence as a neccesary condition
  11. Discussion

Key Result

Proposition 1

Let $G$ denote a set of objects and let $M$ denote a set of properties. If $\mu$ denotes the number of elements in a set of objects then and $\mu$ is an valuation then the lattice is distributive.

Theorems & Definitions (17)

  • Proposition 1
  • Example 2
  • Example 3
  • Example 4
  • Proposition 5
  • Corollary 6
  • Proposition 7
  • Lemma 8
  • Lemma 9
  • Lemma 10: Doob's Maximal Inequalitites Shiryaev1996
  • ...and 7 more