Wave propagation phenomena in nonlinear hierarchical neural networks with predictive coding feedback dynamics

TL;DR

The paper develops a nonlinear, continuous-time model of predictive coding-based hierarchical neural networks and analyzes propagation of activity in bi-infinite and semi-infinite settings. By deriving the time-continuous limit with parameters $p$ and $q$ representing relative feedforward and feedback corrections, it establishes stationary homogeneous states, their linear stability, and the existence of bistable traveling waves with speeds $c_{u\to d}$ and $c_{d\to u}$, including regimes where propagation fails (pinning). It demonstrates threshold phenomena for both constant and flashed external inputs on semi-infinite chains, yielding sharp thresholds $s_0^*$ and $\tau^*$ that delineate propagation from stagnation, with detailed dependence on $(\theta,\mu,p,q)$. The results align with predictive coding theories and offer insights into conditions that could underlie dysfunctional perceptions, while outlining extensions to multi-population networks and adaptation mechanisms for increased biological plausibility and applicability to machine learning.

Abstract

We propose a mathematical framework to systematically explore the propagation properties of a class of continuous in time nonlinear neural network models comprising a hierarchy of processing areas, mutually connected according to the principles of predictive coding. We precisely determine the conditions under which upward propagation, downward propagation or even propagation failure can occur in both bi-infinite and semi-infinite idealizations of the model. We also study the long-time behavior of the system when either a fixed external input is constantly presented at the first layer of the network or when this external input consists in the presentation of constant input with large amplitude for a fixed time window followed by a reset to a down state of the network for all later times. In both cases, we numerically demonstrate the existence of threshold behavior for the amplitude of the external input characterizing whether or not a full propagation within the network can occur. Our theoretical results are consistent with predictive coding theories and allow us to identify regions of parameters that could be associated with dysfunctional perceptions.

Figures (16)

• Figure 1: Illustration of the three branches of stationary homogeneous solutions $x_{d,m,u}(\theta,\mu)$ solutions of $F_p(x)=0$ as a function of $\theta\in(\theta_*(\mu),\theta^*(\mu))$ when condition \ref{['condpmu']} on $\mu$ and $p$ is satisfied. The two branches $x_{d}(\theta,\mu)$ and $x_{u}(\theta,\mu)$ (solid line) are stable while the branch $x_{m}(\theta,\mu)$ (dashed line) is unstable. The so-called bistable regime where the three branches coexist is shaded in violet.
• Figure 2: Space-time plots of the solution $(v_j)_{j\in\mathbb{Z}}$ of \ref{['cont1dZ']} starting from $(h_j^{u\to d})_{j\in\mathbb{Z}}$ (b) or $(h_j^{d\to u})_{j\in\mathbb{Z}}$ (e) in the case where $c_{u\to d}<0$ (a) and $c_{u\to d}>0$ (c) or $c_{d\to u}<0$ (d) and $c_{d\to u}>0$ (f). Values of the parameters were set to $(\theta,\mu,p)=(0.5,16,0.1)$ with $q=0.6$ in (a)-(d) and $q=0.4$ in (c)-(f).
• Figure 3: (a)-(b)-(c) Plots of the wave speeds $c_{u\to d}$ (blue) and $c_{d\to u}$ (light blue) as a function of $\theta\in(\theta_*(\mu),\theta^*(\mu))$ for several values of the parameter $q\in\left\{0.35,0.5,0.65\right\}$. (d)-(e)-(f) Plots of the wave speeds $c_{u\to d}$ (blue) and $c_{d\to u}$ (light blue) as a function of $q\in[0,1]$ for several values of the parameter $\theta\in\left\{0.35,0.5,0.65\right\}$. We observe that at $\theta=\frac{1}{2}$ both wave speeds are equal $c_{u\to d}=c_{d\to u}$ while for $\theta\lessgtr\frac{1}{2}$ one has $c_{d\to u}\lessgtr c_{u\to d}$ for all $q\in[0,1]$. The other parameters $\mu$ and $p$ are set to $\mu=16$ and $p=0.1$.
• Figure 4: (a)-(b) Sign of the wave speeds $c_{u\to d}$ and $c_{d\to u}$ as a function of $q\in[0,1]$ and $\theta\in(\theta_*(\mu),\theta^*(\mu))$. Dark blue region corresponds to positive wave speeds while light blue region corresponds to negative speeds. Region of zero wave speeds is depicted in sky blue. (c) Combined sign of $(c_{u\to d},c_{d\to u})$ as a function of $q\in[0,1]$ and $\theta\in(\theta_*(\mu),\theta^*(\mu))$. The other parameters $\mu$ and $p$ are set to $(\mu,p)=(16,0.1)$.
• Figure 5: Sign of the wave speed $c_{u\to d}$ and $c_{d\to u}$ as a function of $(p,q)\in\left[0,\frac{4}{4+\mu}\right]\times[0,1]$ for several values of the parameter $\theta\in(\theta_*(\mu),\theta^*(\mu))$. Dark blue region corresponds to positive wave speeds while light blue region corresponds to negative speeds. Region of zero wave speeds is depicted in sky blue. The parameter $\mu$ is set to $\mu=16$.

Theorems & Definitions (2)

• Lemma A.1
• Lemma A.2