Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Node Perturbation Can Effectively Train Multi-Layer Neural Networks

Sander Dalm, Marcel van Gerven, Nasir Ahmad

TL;DR

The paper addresses the inefficiency and instability of node perturbation (NP) compared with backpropagation (BP) by recasting NP in terms of directional derivatives, adding input decorrelation, and introducing an activity-based NP (ANP) variant. Decorrelated NP (DNP, DINP, DANP) aligns updates with the true gradient and enables faster convergence, approaching BP performance on CIFAR-10 and robotics tasks, even in noisy or inaccessible-noise hardware. Resampling (multiple noisy passes) scales NP to deeper networks (e.g., Tiny ImageNet) and makes ANP more robust, with DANP showing strong performance in noisy hardware scenarios. Overall, the work demonstrates that decorrelation and directional-derivative framing can render gradient-free NP methods competitive with BP in several practical settings, with significant implications for neuromorphic and on-chip learning.

Abstract

Backpropagation (BP) remains the dominant and most successful method for training parameters of deep neural network models. However, BP relies on two computationally distinct phases, does not provide a satisfactory explanation of biological learning, and can be challenging to apply for training of networks with discontinuities or noisy node dynamics. By comparison, node perturbation (NP), also known as activity-perturbed forward gradients, proposes learning by the injection of noise into network activations, and subsequent measurement of the induced loss change. NP relies on two forward (inference) passes, does not make use of network derivatives, and has been proposed as a model for learning in biological systems. However, standard NP is highly data inefficient and can be unstable due to its unguided noise-based search process. In this work, we develop a modern perspective on NP by relating it to the directional derivative and incorporating input decorrelation. We find that a closer alignment with directional derivatives together with input decorrelation at every layer theoretically and practically enhances performance of NP learning with large improvements in parameter convergence and much higher performance on the test data, approaching that of BP. Furthermore, our novel formulation allows for application to noisy systems in which the noise process itself is inaccessible, which is of particular interest for on-chip learning in neuromorphic systems.

Node Perturbation Can Effectively Train Multi-Layer Neural Networks

TL;DR

The paper addresses the inefficiency and instability of node perturbation (NP) compared with backpropagation (BP) by recasting NP in terms of directional derivatives, adding input decorrelation, and introducing an activity-based NP (ANP) variant. Decorrelated NP (DNP, DINP, DANP) aligns updates with the true gradient and enables faster convergence, approaching BP performance on CIFAR-10 and robotics tasks, even in noisy or inaccessible-noise hardware. Resampling (multiple noisy passes) scales NP to deeper networks (e.g., Tiny ImageNet) and makes ANP more robust, with DANP showing strong performance in noisy hardware scenarios. Overall, the work demonstrates that decorrelation and directional-derivative framing can render gradient-free NP methods competitive with BP in several practical settings, with significant implications for neuromorphic and on-chip learning.

Abstract

Backpropagation (BP) remains the dominant and most successful method for training parameters of deep neural network models. However, BP relies on two computationally distinct phases, does not provide a satisfactory explanation of biological learning, and can be challenging to apply for training of networks with discontinuities or noisy node dynamics. By comparison, node perturbation (NP), also known as activity-perturbed forward gradients, proposes learning by the injection of noise into network activations, and subsequent measurement of the induced loss change. NP relies on two forward (inference) passes, does not make use of network derivatives, and has been proposed as a model for learning in biological systems. However, standard NP is highly data inefficient and can be unstable due to its unguided noise-based search process. In this work, we develop a modern perspective on NP by relating it to the directional derivative and incorporating input decorrelation. We find that a closer alignment with directional derivatives together with input decorrelation at every layer theoretically and practically enhances performance of NP learning with large improvements in parameter convergence and much higher performance on the test data, approaching that of BP. Furthermore, our novel formulation allows for application to noisy systems in which the noise process itself is inaccessible, which is of particular interest for on-chip learning in neuromorphic systems.
Paper Structure (23 sections, 19 equations, 8 figures, 7 tables, 1 algorithm)

This paper contains 23 sections, 19 equations, 8 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Node perturbation and its formulations
  4. Iterative node perturbation
  5. Node perturbation
  6. Activity-based node perturbation
  7. Increasing perturbation learning efficiency through decorrelation
  8. Experimental validation
  9. Results
  10. INP and ANP align better with the true gradient
  11. Decorrelation improves convergence
  12. Multi-layer networks can be trained effectively with node perturbation
  13. Resampling allows for scaling to larger systems
  14. Robot control task
  15. Node perturbation allows learning in noisy systems
  16. ...and 8 more sections

Figures (8)

  • Figure 1: Left: Angles between BP's weight update and an update calculated by NP, ANP and INP as a function of the number of noise iterations. Note that the network is not updated across the 'Noise iterations' dimension, but instead multiple noise samples are propagated through a fixed network and their updates averaged. This is computed for a fully-connected, 3 hidden layer, network with leaky ReLU nonlinearities. Right: Number of forward passes for NP, ANP and INP.
  • Figure 2: Performance of (D)NP and (D)BP on CIFAR-10 when training a single-layer architecture.
  • Figure 3: Performance of (D)NP, (D)ANP, (D)INP and (D)BP on CIFAR-10 in a three-hidden layer network. For both figures, curves report mean train and test accuracy. Shaded areas indicate the maximal and minimal accuracy obtained for three random seeds. Note that all NP methods have an equivalent formulation in a single-layer network.
  • Figure 4: Train and test accuracy for BP, DNP, DANP and DINP on Tiny ImageNet. DANP and DINP applied with 1, 10, 30 and 100 noise samples per update step. DNP applied with 100 noise samples per update step.
  • Figure 5: Train and test loss for (D)NP, (D)ANP and (D)BP on the SARCOS dataset. Curves report mean train and test loss. Error bars indicate the maximal and minimal loss obtained for three random seeds.
  • ...and 3 more figures