Scaling of hardware-compatible perturbative training algorithms

TL;DR

This paper introduces multiplexed gradient descent (MGD), a scalable, hardware-friendly perturbative method for estimating gradients without full backpropagation, extended to both weight and node perturbations. It demonstrates that the gradient-estimation time grows with network size, but the time to reach a specified accuracy does not necessarily scale linearly and can even improve for larger networks, enabling training of networks with over a million parameters on neuromorphic hardware. By proving that the MGD gradient estimator is unbiased and compatible with momentum and Adam, the authors show that standard optimization practices can be applied in a zero-order, hardware-centric setting, achieving accuracy comparable to backpropagation. The work provides a practical pathway toward efficient, in-situ training of large-scale neural networks on neuromorphic and analog hardware, with hardware-aware tunings like the perturbation time constant $\tau_\theta$ and perturbation type (weight vs node) to balance update costs and convergence speed.

Abstract

In this work, we explore the capabilities of multiplexed gradient descent (MGD), a scalable and efficient perturbative zeroth-order training method for estimating the gradient of a loss function in hardware and training it via stochastic gradient descent. We extend the framework to include both weight and node perturbation, and discuss the advantages and disadvantages of each approach. We investigate the time to train networks using MGD as a function of network size and task complexity. Previous research has suggested that perturbative training methods do not scale well to large problems, since in these methods the time to estimate the gradient scales linearly with the number of network parameters. However, in this work we show that the time to reach a target accuracy--that is, actually solve the problem of interest--does not follow this undesirable linear scaling, and in fact often decreases with network size. Furthermore, we demonstrate that MGD can be used to calculate a drop-in replacement for the gradient in stochastic gradient descent, and therefore optimization accelerators such as momentum can be used alongside MGD, ensuring compatibility with existing machine learning practices. Our results indicate that MGD can efficiently train large networks on hardware, achieving accuracy comparable to backpropagation, thus presenting a practical solution for future neuromorphic computing systems.

Figures (6)

• Figure 1: (a) Illustration of weight perturbation, where each weight is perturbed individually. Weights can be updated entirely within the synapse by combining the global cost-feedback signal with the local perturbation. (b) Illustration of node perturbation, where only the summed input to each neuron is perturbed. Here, weight updates must be performed through a one-step backpropagation process that computes the error at the neuron input, and then passes it backwards (through a multiplication) to the synapse.
• Figure 2: Illustration of gradient descent down a cost landscape using MGD, for varying amounts of averaging. (a) In MGD, the true gradient (black dashed line) is not presumed to be known analytically, and is instead estimated. The gradient estimation (red-shaded regions) starts out inaccurate but can be refined to arbitrary precision with additional averaging toward the true gradient. (b) The resulting weight updates and trajectory down the cost-landscape (red lines) do not strictly follow the gradient, but instead deviate to a degree that depends on the amount of averaging.
• Figure 3: Analyzing the behavior of gradient estimation in MGD. (a) The accuracy of the gradient estimation versus time. Each iteration refines the the accuracy of the gradient estimation, defined as the cosine of the angle ($\alpha$) between the true local gradient $\mathbf{\nabla_{\bf W}} \text{C}$ and local gradient estimated by MGD ${\bf G}\xspace$ for a network with $N$$=4.52\times10^5$ parameters. (b) Number of iterations it takes for $\cos(\alpha)$ to reach 0.95 versus total number of trained parameters in the network ($N$). Dashed lines represent the expected $N$ and $\sqrt{N\xspace}$ scaling for weight nad node perturbation, respectively. (c) Number of iterations it takes for $\cos(\alpha)$ to reach 0.95 (or 95% gradient estimation accuracy) versus the task complexity. The shaded regions represent the upper and lower quartile bounds of 10 random initializations, and the solid line corresponds to the median.
• Figure 4: Time needed to successfully train networks with MGD as a function of network size and task complexity (a) Testing error versus number of iterations for network with $N\xspace=2.55\times 10^4$ parameters, showing that the MGD algorithm can match the same final accuracy of 91.6% as backpropagation. Although backpropagation is generally not available in hardware, here in simulation its analytical nature provides the fastest time-to-train as expected. (b) Number of iterations it takes for the network to be trained to 80% testing accuracy versus total number of trained parameters in the network N. (c) Number of iterations it takes for the network to be trained to 80% testing accuracy versus the task complexity.
• Figure 5: Number of weight updates required to train FashionMNIST on a $2.55\times10^5$ parameter network to 80% testing accuracy versus $\tau_\uptheta$.

Theorems & Definitions (4)

• Theorem 1
• proof
• Corollary 1.1
• proof