Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Synthesizable RTL Implementation of Predictive Coding Networks

Timothy Oh

Abstract

Backpropagation has enabled modern deep learning but is difficult to realize as an online, fully distributed hardware learning system due to global error propagation, phase separation, and heavy reliance on centralized memory. Predictive coding offers an alternative in which inference and learning arise from local prediction-error dynamics between adjacent layers. This paper presents a digital architecture that implements a discrete-time predictive coding update directly in hardware. Each neural core maintains its own activity, prediction error, and synaptic weights, and communicates only with adjacent layers through hardwired connections. Supervised learning and inference are supported via a uniform per-neuron clamping primitive that enforces boundary conditions while leaving the internal update schedule unchanged. The design is a deterministic, synthesizable RTL substrate built around a sequential MAC datapath and a fixed finite-state schedule. Rather than executing a task-specific instruction sequence inside the learning substrate, the system evolves under fixed local update rules, with task structure imposed through connectivity, parameters, and boundary conditions. The contribution of this work is not a new learning rule, but a complete synthesizable digital substrate that executes predictive-coding learning dynamics directly in hardware.

A Synthesizable RTL Implementation of Predictive Coding Networks

Abstract

Backpropagation has enabled modern deep learning but is difficult to realize as an online, fully distributed hardware learning system due to global error propagation, phase separation, and heavy reliance on centralized memory. Predictive coding offers an alternative in which inference and learning arise from local prediction-error dynamics between adjacent layers. This paper presents a digital architecture that implements a discrete-time predictive coding update directly in hardware. Each neural core maintains its own activity, prediction error, and synaptic weights, and communicates only with adjacent layers through hardwired connections. Supervised learning and inference are supported via a uniform per-neuron clamping primitive that enforces boundary conditions while leaving the internal update schedule unchanged. The design is a deterministic, synthesizable RTL substrate built around a sequential MAC datapath and a fixed finite-state schedule. Rather than executing a task-specific instruction sequence inside the learning substrate, the system evolves under fixed local update rules, with task structure imposed through connectivity, parameters, and boundary conditions. The contribution of this work is not a new learning rule, but a complete synthesizable digital substrate that executes predictive-coding learning dynamics directly in hardware.
Paper Structure (38 sections, 15 equations, 3 figures, 1 table)

This paper contains 38 sections, 15 equations, 3 figures, 1 table.

Table of Contents

  1. Artifact availability.
  2. Introduction
  3. Contributions.
  4. Related Work
  5. Neuromorphic and local-learning hardware.
  6. Predictive coding as hierarchical inference.
  7. Relationship to backpropagation.
  8. Incremental predictive coding and stability.
  9. Backpropagation as a biological and hardware mismatch.
  10. Backpropagation and Its Constraints
  11. Predictive Coding
  12. Activity dynamics and synaptic updates
  13. Component-wise form
  14. Discrete-time update implemented in hardware
  15. A Digital Predictive Coding Substrate
  16. ...and 23 more sections

Figures (3)

  • Figure 1: Training curve for the $2 \rightarrow 4 \rightarrow 3$ ReLU network. After a brief transient at epoch $1$, MSE descends rapidly then settles into a slow-improvement plateau.
  • Figure 2: Training curve for the $2 \rightarrow 2 \rightarrow 1$ tanh network. MSE drops more than two orders of magnitude by epoch $3$ and subsequently improves slowly to a small residual.
  • Figure 3: Training curves for three architectures of increasing dimension. All exhibit rapid initial descent followed by a stable residual floor that rises modestly with network size.