Minimal thermodynamic cost of communication

TL;DR

This work addresses the thermodynamic cost of communication by formulating a universal lower bound on entropy production per channel use via mismatch cost (MMC), valid for any channel dynamics. The authors show that the MMC per use satisfies $\mathrm{MC}(p_X) \ge \mathrm{I}(X;Y)$ and, under a specific MMC decomposition, relate minimal cost to the information rate and chosen priors, independent of microscopic details. They extend the analysis to encoding and decoding using the periodic-machine framework, deriving stepwise MMC bounds for linear encoders and syndrome decoders and demonstrating end-to-end trade-offs between block error rate and total MMC. A binary-channel example reveals concave and non-concave MMC–information relationships depending on priors, illustrating reverse multiplexing as a possible strategy to reduce total thermodynamic cost. Overall, the paper provides a principled, information-theoretic view of energy costs in computation and communication with potential implications for biology and neuromorphic engineering.

Abstract

Thermodynamic cost of communication is a major factor in the thermodynamic cost of real-world computers, both biological and digital. Despite its importance, the fundamental principles underlying this cost remain poorly understood. This paper makes two major contributions to addressing this gap. First, we derive a universal relationship between information transmission rate and minimal entropy production (EP) by focusing on the mismatch cost (MMC) component of thermodynamic cost. The resulting relationship holds independently of the underlying physical dynamics, making it broadly applicable. We discuss the implications of the derived minimal communication cost for work extraction in measurement-and-feedback protocols, and through examples involving binary channels, we show that the relationship between transmission rate and minimal thermodynamic cost can exhibit diminishing returns in certain scenarios. Second, we extend this thermodynamic analysis to the computational front and back ends critical to communication-namely, encoding and decoding to reduce errors in noisy transmission. Using the framework of periodic machines, we establish strictly positive minimal costs for implementing linear error-correcting codes. We compare these costs with end-to-end error rates, highlighting trade-offs between thermodynamic cost and decoding accuracy.

Figures (16)

• Figure 1: Schematic of a communication channel: (i) The state of the input node $X$ is updated with a new value drawn from the information source. (ii) This input state is then copied to the output node $Y$, subject to noise characterized by the conditional distribution $\pi_{Y|X}$.
• Figure 2: Mismatch cost increases significantly with entropy flow scaling $k$, demonstrating its dominant contribution in high-EP scenarios. Starting with arbitrary chosen values $f(1) = 0.5$, $f(2) = 0.1$, and $f(3) = 0.2$, we scale all $f(x)$ by a factor $k$, resulting in new values $k f(x)$. (a) As $k$ increases, the associated prior distribution $q_{X_0} = \mathrm{arg}\min_{r_{X_0}} \mathcal{C}(r_{X_0})$ (as defined by Eq. \ref{['EPdef2']}) shifts closer to the boundary of the 2-simplex. (b) Consequently, the mismatch cost for a typical initial distribution (here, $p_{X_0} = \{0.46, 0.33, 0.21\}$) becomes a larger fraction of the total EP, eventually exceeding $60\%$ of the total EP. The calculations are performed using the map $G$ defined in Eq. \ref{['eq:mapG']}, with $\phi$ set to 0.1. The total entropy production is computed from $f(x)$ and $f(x)$ using Eq. \ref{['EPdef2']}, while the corresponding mismatch cost for each resulting prior is calculated using Eq. \ref{['eq:MMCdef']}.
• Figure 3: We model the communication channel as consisting of an input node $X$ and an output node $Y$. At each iteration, a value is sampled from the input distribution $p_X$ and copied to the output according to the conditional distribution $\pi_{Y|X}$. Following this copy operation, the input is updated with a new value, independently drawn from $p_X$, thereby resetting the joint state of $(X, Y)$ from correlated to uncorrelated.
• Figure 4: Diagram illustrating how the joint distribution of the channel evolves during (a) the overwriting of the input and (b) the noisy copying of the input state to the output. Here, $p^a_{XY}(x, y) = p_X(x)p_Y(y)$ and $p^b_{XY}(x, y) = \pi_{Y|X}(y|x)p_X(x)$. With each use of the channel, the joint distribution cycles from $p^b_{XY}$ to $p^a_{XY}$ and back again.
• Figure 5: Diagram illustrating how the prior distributions associated with (a) the overwriting process and (b) the noisy copying of the input state to the output evolve through their respective transformations. In (a), the prior changes from $q^{a}_{XY}(x, y) = q^{A}(x) q^{a}_Y(y)$ to $\Tilde{q}^a_{XY}(x, y) = p_X(x) q^a_Y(y)$. In (b), the prior changes from $q^{b}_{XY}(x, y)$ to $\Tilde{q}^b_{XY}(x, y) = \pi_{Y|X}(y|x)q^b_X(x)$.