Mutual Information of a class of Poisson-type Channels using Markov Renewal Theory

TL;DR

The results of classical (Markov renewal) filtering theory in a novel manner are applied in a novel manner to the problem of exactly computing the MI/MIR.

Abstract

The mutual information (MI) of Poisson-type channels has been linked to a filtering problem since the 70s, but its evaluation for specific continuous-time, discrete-state systems remains a demanding task. As an advantage, Markov renewal processes (MrP) retain their renewal property under state space filtering. This offers a way to solve the filtering problem analytically for small systems. We consider a class of communication systems $X \to Y$ that can be derived from an MrP by a custom filtering procedure. For the subclasses, where (i) $Y$ is a renewal process or (ii) $(X,Y)$ belongs to a class of MrPs, we provide an evolution equation for finite transmission duration $T>0$ and limit theorems for $T \to \infty$ that facilitate simulation-free evaluation of the MI $\mathbb{I}(X_{[0,T]}; Y_{[0,T]})$ and its associated mutual information rate (MIR). In other cases, simulation cost is reduced to the marginal system $(X,Y)$ or $Y$. We show that systems with an additional $X$-modulating level $C$, which statically chooses between different processes $X_{[0,T]}(c)$, can naturally be included in our framework, thereby giving an expression for $\mathbb{I}(C; Y_{[0,T]})$. Our primary contribution is to apply the results of classical (Markov renewal) filtering theory in a novel manner to the problem of exactly computing the MI/MIR. The theoretical framework is showcased in an application to bacterial gene expression, where filtering is analytically tractable.

Figures (4)

• Figure 1: The synthetic biology scenario; a) shows the transition graph of the model of interest. Transitions into $J$ are equivalent to arrivals at $Y_{}$. State $J$ mirrors state $P_{\mathrm{on}}$ in terms of outgoing transitions, such that being in state $J$ is equivalent to being in $P_{\mathrm{on}}$. Transitions into $P_{\mathrm{off}}$ model off-switching of $X_{}$ and transitions $P_{\mathrm{off}} \to P_{\mathrm{on}}$ model on-switching; b) analytically obtained inter-arrival time densities of $Y_{}$ for both outcomes of $C{}$; c)$\mathbb{I}\left(C{};\,Y_{\left[0,\,T\right]}\right)$ for different success probabilities $\pi$ and different $T$ for $10^5$ trajectories of $Y_{}$. The dashed red line highlights the value of $\pi$ for attained maximal $\mathbb{I}\left(C{};\,Y_{\left[0,\,T_{\max}\right]}\right)$. The kinetic parameters are listed in section \ref{['sec:Application']}.
• Figure 2: Representations of simple gene expression model with a leaky two-state promoter as the input $X_{}$, where $X_{}=1$ denotes the active state and $X_{}=r \in (0,1)$ is the leaky inactive state with $r$ being the fraction of activity in the inactive state. The output $Y_{}$ models mRNA synthesis events. a) models the output directly as a counting process such that the state space is infinite, while b) represents the model on a finite state space with two jump states, $J_1$ and $J_r$.
• Figure 3: State transition diagram of a class of semi-Markov processes representing the communication system $(X_{}, Y_{})$. Being in one of the yellow states means $X_{t}=1$, whereas being in a blue state means $X_{t}=0$. Transitions into $\mathrm{J}$ (orange edges) account for arrivals at $Y_{}$. The marginal process $Y_{}$ is a renewal process for this class.
• Figure 4: Visualization of the state space augmentation for the case study; a) shows the transition graph of the model of interest with transition class indices (blue) assigned to the transitions; b) depicts the corresponding augmented state space.

Theorems & Definitions (9)

• Theorem 1: Regenerative form of the intensity bremaud1981point
• Theorem 2: Anderson's filtering theorem for MrPs cinlar1969markov
• Theorem 3
• Proposition 1
• Theorem 4
• Definition 1
• Proposition 2
• Proposition 3
• Lemma 1: Markov renewal equations cinlar1969markov