Principle of Information Increase: An Operational Perspective of Information Gain in the Foundations of Quantum Theory

TL;DR

This work addresses how to quantify information gain from quantum measurements by distinguishing two continuous-entropy-based measures, differential information gain $I_{ ext{diff}}$ and relative information gain $I_{ ext{rel}}$, and by introducing the Principle of Information Increase to guide metric and prior choice. Using a Bayesian framework with a beta prior for two-outcome systems, the authors derive explicit finite-$N$ expressions and large-$N$ approximations for $I_{ ext{diff}}$ and $I_{ ext{rel}}$, showing that $I_{ ext{rel}}$ is always non-negative while $I_{ ext{diff}}$ can be negative depending on the prior. They prove that the expected information gain in the next measurement equals both $\overline{I}_{\text{diff}}$ and $\overline{I}_{\text{rel}}$, and they demonstrate that Jeffreys' binomial prior ($\alpha=-\tfrac{1}{2}$) yields maximal robustness of $I_{ ext{diff}}$ to data sequences, with asymptotic behavior $\overline{I}=\tfrac{1}{2N}$. The paper argues for adopting differential information gain with the Jeffreys prior in two-outcome quantum systems, and connects these results to prior work by Summhammer, Goyal, and Wootters, highlighting implications for information-based reconstructions of quantum theory and measurement design.

Abstract

A measurement performed on a quantum system is an act of gaining information about its state, a view that is widespread in practical and foundational work in quantum theory. However, the concept of information in quantum theory reconstructions is multiply-defined, and its conceptual foundations remain surprisingly under-explored. In this paper, we investigate the gain of information in quantum measurements from an operational viewpoint. We show that the continuous extension of the Shannon entropy naturally admits two distinct measures of information gain, differential information gain and relative information gain, and that these have radically different characteristics. In particular, while differential information gain can increase or decrease as additional data is acquired, relative information gain consistently grows, and moreover exhibits asymptotic indifference to the data or choice of Bayesian prior. In order to make a principled choice between these measures, we articulate a Principle of Information Increase, which incorporates Summhammer's proposal that more data from measurements leads to more knowledge about the system, and also takes into consideration black swan events. This principle favors differential information gain as the more relevant metric in two-outcome quantum systems, and guides the selection of priors for these information measures. Finally, we show that, of the beta distribution priors, the Jeffreys' binomial prior is the prior ensures maximal robustness of information gain to the particular data sequence obtained in a run of experiments.

Figures (10)

• Figure 1: Differential Information Gain in a Single Toss. Assuming we have data from the first $N$ tosses, denoted as $T_N$. Using a specific prior distribution, we can calculate the information gain for these first $N$ tosses, denoted as $I(N)$. If we now consider the $(N+1)$th toss and obtain the result $t_{N+1}$, we can repeat the same procedure to calculate the information gain for a total of $N+1$ tosses, denoted as $I(N+1)$. The information gain specific to the $(N+1)$th toss can be obtained as the difference between $I(N+1)$ and $I(N)$.
• Figure 2: Relative Information Gain in a Single Toss. The posterior distribution calculated from the results of the first $N$ tosses serves as the prior for the $(N+1)$th toss. The KL divergence between this posterior and the subsequent posterior represents the information gain in the $(N+1)$th toss.
• Figure 3: Differential Information Gain ($I_{\text{diff}}$) vs. $N$ for Different Priors. Here, the $y$-axis represents the value of $I_{\text{diff}}$, and the $x$-axis corresponds to the value of $N$. In each graph, we fix the value of $\alpha$ to allow for a comparison of the behavior of $I_{\text{diff}}$ under different priors. Given $N$, there are $N+1$ points as $h_N$ ranges from $0$ to $N$. Notably, for $\alpha = -0.7$, all points lie above the $x$-axis, while for other priors, negative points are present and the fraction of negative points becomes constant as $N$ increases. The asymptotic behavior of this fraction will be shown in Figure \ref{['FoNvsN_graph']}. Moreover, it appears that the graph is most concentrated when $\alpha = -0.5$, whereas for $\alpha < -0.5$ and $\alpha > -0.5$, the graph becomes more dispersed. This dispersive/concentrating feature is clearly depicted in Figure \ref{['dig_stddev']}.
• Figure 4: Fraction of Negatives (FoN) vs. $N$ under different $\alpha$. In Figure \ref{['dig_graph']}, we can observe that larger $\alpha$ values lead to more dispersed lines and an increased number of negative values for each $N$. We use FoN to quantify this fraction of negative points. It appears that for $\alpha \le -0.7$, FoN is consistently zero, indicating that $I_{\text{diff}}$ is always positive. For $\alpha \le -0.5$ FoN decreases and tends to be zero as $N$ becomes large, while for $\alpha > -0.5$, FoN tends to a constant as $N$ increases, and this constant grows with increasing $\alpha$.
• Figure 5: Fraction of Negatives (FoN) vs. $\alpha$ for Different $N$. We identify a critical point, denoted as $\alpha_p$, where the FoN equals zero when $\alpha \le \alpha_p$. The critical point exhibits a gradual variation with respect to $N$, following these patterns: (i) for small $N$, $\alpha_p$ is close proximity to $-0.68$; (ii) for large $N$, $\alpha_p$ tends to $-0.5$.