Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conserved active information

Yanchen Chen, Daniel Andrés Díaz-Pachón

TL;DR

To address NFLT-driven limitations of average information measures, the paper introduces active information $I^+$ and its conservation counterpart $I^\oplus$. It provides a measure-theoretic formulation of $I^+$ and derives a closed-form baseline regime for $I^\oplus$ under a uniform baseline, showing when external information is needed versus when global order can emerge through internal redistribution; The results are illustrated with Bernoulli distributions, Markov chains, and cosmological fine-tuning and are connected to potential applications in search, optimization, AI, and estimation. Overall, the work offers a principled framework to quantify information balance across the entire search space, extending beyond KL and providing a tool for diagnosing when problem-specific knowledge yields net conservation or requires external input.

Abstract

We introduce conserved active information $I^\oplus$, a symmetric extension of active information that quantifies net information gain/loss across the entire search space, respecting No-Free-Lunch conservation. Through Bernoulli and uniform-baseline examples, we show $I^\oplus$ reveals regimes hidden from KL divergence, such as when strong knowledge reduces global disorder. Such regimes are proven formally under uniform baseline, distinguishing disorder (increasing mild knowledge from order-imposing strong knowledge. We further illustrate these regimes with examples from Markov chains and cosmological fine-tuning. This resolves a longstanding critique of active information while enabling applications in search, optimization, and beyond.

Conserved active information

TL;DR

To address NFLT-driven limitations of average information measures, the paper introduces active information and its conservation counterpart . It provides a measure-theoretic formulation of and derives a closed-form baseline regime for under a uniform baseline, showing when external information is needed versus when global order can emerge through internal redistribution; The results are illustrated with Bernoulli distributions, Markov chains, and cosmological fine-tuning and are connected to potential applications in search, optimization, AI, and estimation. Overall, the work offers a principled framework to quantify information balance across the entire search space, extending beyond KL and providing a tool for diagnosing when problem-specific knowledge yields net conservation or requires external input.

Abstract

We introduce conserved active information , a symmetric extension of active information that quantifies net information gain/loss across the entire search space, respecting No-Free-Lunch conservation. Through Bernoulli and uniform-baseline examples, we show reveals regimes hidden from KL divergence, such as when strong knowledge reduces global disorder. Such regimes are proven formally under uniform baseline, distinguishing disorder (increasing mild knowledge from order-imposing strong knowledge. We further illustrate these regimes with examples from Markov chains and cosmological fine-tuning. This resolves a longstanding critique of active information while enabling applications in search, optimization, and beyond.
Paper Structure (4 sections, 4 theorems, 18 equations, 4 figures, 1 table)

This paper contains 4 sections, 4 theorems, 18 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Active information
  3. Conserved active information
  4. Discussion

Key Result

Lemma 1

Let $\mathcal{X} = \mathcal{X}_1 \cup \mathcal{X}_2$, $\mathcal{F} = \sigma(\mathcal{F}_1, \mathcal{F}_2)$, and extend the measures $\mathbf P_i$, for $i=1,2$, to Then the measurable space $(\mathcal{X}, \mathcal{F})$ and the probability measures $\mathbf P_1^*, \mathbf P_2^*$ associated with it exist and are well-defined.

Figures (4)

  • Figure 1: Total information $\mathcal{H}(X)$ of $X\sim\textrm{Ber}(p)$. Logs taken in base 2.
  • Figure 2: Entropy $H(X)$ of $X\sim\textrm{Ber}(p)$. Logs taken in base 2
  • Figure 3: $I^\oplus(\mathbf P_1,\mathbf P_2)$ when $\mathbf P_1 \sim\mathrm{Ber}(p)$ and $\mathbf P_2 \sim\mathrm{Ber}(q)$.
  • Figure 4: $\mathrm{KL}\divx{\mathbf P_1}{\mathbf P_2}$ when $\mathbf P_1 \sim\mathrm{Ber}(p)$ and $\mathbf P_2 \sim\mathrm{Ber}(q)$.

Theorems & Definitions (13)

  • Lemma 1
  • proof
  • Remark 1
  • Definition 1
  • Example 1
  • Lemma 2
  • Example 2
  • Theorem 3: Regimes of Conserved Active Information under Uniform Baseline
  • proof
  • Remark 2
  • ...and 3 more