Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heterogeneous Time Constants Improve Stability in Equilibrium Propagation

Yoshimasa Kubo, Suhani Pragnesh Modi, Smit Patel

TL;DR

HTS is introduced for EP by assigning neuron-specific time constants drawn from biologically motivated distributions that improves training stability while maintaining competitive task performance and suggests that incorporating heterogeneous temporal dynamics enhances both the biological realism and robustness of equilibrium propagation.

Abstract

Equilibrium propagation (EP) is a biologically plausible alternative to backpropagation for training neural networks. However, existing EP models use a uniform scalar time step dt, which corresponds biologically to a membrane time constant that is heterogeneous across neurons. Here, we introduce heterogeneous time steps (HTS) for EP by assigning neuron-specific time constants drawn from biologically motivated distributions. We show that HTS improves training stability while maintaining competitive task performance. These results suggest that incorporating heterogeneous temporal dynamics enhances both the biological realism and robustness of equilibrium propagation.

Heterogeneous Time Constants Improve Stability in Equilibrium Propagation

TL;DR

HTS is introduced for EP by assigning neuron-specific time constants drawn from biologically motivated distributions that improves training stability while maintaining competitive task performance and suggests that incorporating heterogeneous temporal dynamics enhances both the biological realism and robustness of equilibrium propagation.

Abstract

Equilibrium propagation (EP) is a biologically plausible alternative to backpropagation for training neural networks. However, existing EP models use a uniform scalar time step dt, which corresponds biologically to a membrane time constant that is heterogeneous across neurons. Here, we introduce heterogeneous time steps (HTS) for EP by assigning neuron-specific time constants drawn from biologically motivated distributions. We show that HTS improves training stability while maintaining competitive task performance. These results suggest that incorporating heterogeneous temporal dynamics enhances both the biological realism and robustness of equilibrium propagation.
Paper Structure (6 sections, 3 equations, 1 figure, 1 table)

This paper contains 6 sections, 3 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Model Dynamics
  4. Heterogeneous Time Steps
  5. Experimental Setup
  6. Results and Conclusion

Figures (1)

  • Figure 1: Distributions of heterogeneous time steps ($dt_i$) used in the hidden layer. All distributions were parameterized with $\mu=0.3$, $\sigma=0.1$ and clamped to $[10^{-3}, 0.5]$ for numerical stability. The visible accumulation at $dt=0.5$ for heavy-tailed distributions results from this truncation.