Entropy production in communication channels

TL;DR

The paper builds a minimal, physics-based framework that ties thermodynamic costs to information transmission in far-from-equilibrium channels by merging stochastic thermodynamics with Shannon information theory. It analyzes two core channel-typing schemes—energy switching and reservoir switching—showing that the entropy production rate $\langle \dot{\sigma} \rangle$ is not universally a monotonic or convex function of channel capacity $C$, though convexity emerges at high $C$. It derives a minimax bound on $C$, proposes a Pareto-front construction for distributing information across multiple channels, and demonstrates, via two- and multi-channel analyses, that inverse multiplexing can reduce thermodynamic costs under specific regimes. The results illuminate when splitting a information stream across multiple channels is thermodynamically advantageous and offer guidance for designing energy-efficient communication in biological and artificial systems.

Abstract

In many complex systems, whether biological or artificial, the thermodynamic costs of communication among their components are large. These systems also tend to split information transmitted between any two components across multiple channels. A common hypothesis is that such inverse multiplexing strategies reduce total thermodynamic costs. So far, however, there have been no physics-based results supporting this hypothesis. This gap existed partially because we have lacked a theoretical framework that addresses the interplay of thermodynamics and information in off-equilibrium systems. Here we present the first study that rigorously combines such a framework, stochastic thermodynamics, with Shannon information theory. We develop a minimal model that captures the fundamental features common to a wide variety of communication systems, and study the relationship between the entropy production of the communication process and the channel capacity, the canonical measure of the communication capability of a channel. In contrast to what is assumed in previous works not based on first principles, we show that the entropy production is not always a convex and monotonically increasing function of the channel capacity. However, those two properties are recovered for sufficiently high channel capacity. These results clarify when and how to split a single communication stream across multiple channels.

Figures (6)

• Figure 1: Examples of communication channels include a) neuronal synapses, b) CMOS inverters, or c) adjacent animals in a flock. These communication channels effectively d) consist of two nodes: an input A and an output B, which attempts to copy the state of A. The conditional probability distribution of the output state given the input state, $p(X_B | X_A)$, reflects the noisiness of the copying process. Since this process must occur out of equilibrium, it requires external work at a rate $\dot{W}$ and dissipates some amount of that work in terms of entropy flow to the surrounding environment at a rate $\dot{S}_e$. e) Such a communication channel can be modeled as a stochastic process. For a fixed time period of length $t$, the system can have many different realizations, or trajectories $\vec{x}$. These trajectories capture how the state $x_A$ of the input and the state $x_B$ of the output co-evolve. In each trajectory, the output's attempts to copy the state of the input result in an entropy production (EP) $\sigma(\vec{x})$. Boxes highlighted in green represent the time periods for which the state of the output matches the state of the input. Note that there is stochasticity both in the values of the states as well as in the timing of the state transitions. f) Averaged over all possible trajectories, the state occupancies of the input and output define the conditional distribution $p(X_B | X_A)$, and an average EP $\langle \sigma \rangle$ of the process. Typically the average EP rate, $\langle \dot{\sigma} \rangle$, accounts for most of the rate of entropy flow to the environment, $\langle \dot{S}_e \rangle$.
• Figure 2: Entropy production versus channel capacity for the channel given in Eq. \ref{['eq:simple_channel']}.
• Figure 3: Efficiency versus channel capacity for the channel given in Eq. \ref{['eq:simple_channel']}.
• Figure 4: Results for the case when the state energies of the communication channel's output vary with the input state. Here the output couples to two reservoirs with temperatures $T_1$ and $T_2$. We vary the noise in the channel by holding the temperature of one reservoir fixed and varying the temperature of the other reservoir. Black dots in each graph indicate $T_1 = T_2$. (a) These plots reflect the case of a binary alphabet ($L=2$) and for energy bias $\epsilon = 2$. (a, top left) We first analyze the case of zero input signal rate: $f_s = 0$. We find that the EP rate $\langle \dot{\sigma} \rangle = \langle \dot{\sigma}_B \rangle$ as a function of $T_1$ has a single global root when $T_1 = T_2$. The derivative of the EP rate is negative (non-negative) to the left (right) of this root. (a, top right) The channel capacity $C$ is a positive function of $T_1$ and its derivative is non-positive. (a, bottom) Combining these two relationships, we find that the EP rate has at most one minimum with respect to the channel capacity. As a result, $\langle \dot{\sigma} \rangle(C)$ is a convex and non-monotonic function (see proof in Materials and Methods). (b) Plots of the function $\langle \dot{\sigma}_B \rangle(C)$ for (top) different signaling rates $f_s > 0$, (middle) different alphabet lengths $L$, and (bottom) different energy biases $\epsilon$. We observe in all plots that the EP rate retains a single global minimum (marked by black dots) with respect to the channel capacity.
• Figure 5: Entropy production rate versus channel capacity for the minimal model of reservoir switching. The parameters are $T_1 /\epsilon = 0.1$, $\beta(1)=\beta(0)^{-1} = 0.1$, and $M=4$. The temperature $T_2$ was varied in the range $(10^{-1}, 10^2)$ to obtain each of the curves.