Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Artificial Neural Microcircuits as Building Blocks: Concept and Challenges

Andrew Walter, Shimeng Wu, Andy M. Tyrrell, Liam McDaid, Malachy McElholm, Nidhin Thandassery Sumithran, Jim Harkin, Martin A. Trefzer

TL;DR

How large neural networks, particularly Spiking Neural Networks (SNNs) can be assembled using Artificial Neural Microcircuits (ANMs), intended as off-the-shelf components, is articulated and the results of initial work to produce a catalogue of such Microcircuits though the use of Novelty Search are shown.

Abstract

Artificial Neural Networks (ANNs) are one of the most widely employed forms of bio-inspired computation. However the current trend is for ANNs to be structurally homogeneous. Furthermore, this structural homogeneity requires the application of complex training and learning tools that produce application specific ANNs, susceptible to pitfalls such as overfitting. In this paper, an new approach is explored, inspired by the role played in biology by Neural Microcircuits, the so called ``fundamental processing elements'' of organic nervous systems. How large neural networks, particularly Spiking Neural Networks (SNNs) can be assembled using Artificial Neural Microcircuits (ANMs), intended as off-the-shelf components, is articulated; the results of initial work to produce a catalogue of such Microcircuits though the use of Novelty Search is shown; followed by efforts to expand upon this initial work, including a discussion of challenges uncovered during these efforts and explorations of methods by which they might be overcome.

Artificial Neural Microcircuits as Building Blocks: Concept and Challenges

TL;DR

How large neural networks, particularly Spiking Neural Networks (SNNs) can be assembled using Artificial Neural Microcircuits (ANMs), intended as off-the-shelf components, is articulated and the results of initial work to produce a catalogue of such Microcircuits though the use of Novelty Search are shown.

Abstract

Artificial Neural Networks (ANNs) are one of the most widely employed forms of bio-inspired computation. However the current trend is for ANNs to be structurally homogeneous. Furthermore, this structural homogeneity requires the application of complex training and learning tools that produce application specific ANNs, susceptible to pitfalls such as overfitting. In this paper, an new approach is explored, inspired by the role played in biology by Neural Microcircuits, the so called ``fundamental processing elements'' of organic nervous systems. How large neural networks, particularly Spiking Neural Networks (SNNs) can be assembled using Artificial Neural Microcircuits (ANMs), intended as off-the-shelf components, is articulated; the results of initial work to produce a catalogue of such Microcircuits though the use of Novelty Search is shown; followed by efforts to expand upon this initial work, including a discussion of challenges uncovered during these efforts and explorations of methods by which they might be overcome.
Paper Structure (25 sections, 9 equations, 30 figures, 3 tables, 1 algorithm)

This paper contains 25 sections, 9 equations, 30 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Neural Microcircuits
  3. Artificial Neural Microcircuits
  4. Methodology
  5. Generating Prospective Microcircuits
  6. Microcircuit Connection Matrices
  7. Behavioral Assessment
  8. Catalogue Population
  9. Proof-of-Concept Experiment
  10. Stimulus
  11. Parameters
  12. Results
  13. Expanded Search Experiment
  14. Adjusted Motif Selection
  15. Pruning Algorithm
  16. ...and 10 more sections

Figures (30)

  • Figure 1: An illustration of the various approaches to neural network topologies, including the approach proposed in this paper: a) Feed Forward; b) Recurrent; c) Evolved from Scratch; d) Microcircuit based.
  • Figure 2: The crayfish ventral nerve cord (a), taken from Demyanenko-2019; with a block illustration of the ganglia of the abdominal section (b), adapted from Smarandache-Wellmann-2014
  • Figure 3: Illustration of the Microcircuit within the left hemiganglion of segment A4, counterparts of which are repeated in the segments A2 through A5. Arrows are connections to other microcircuits, dot lines are biochemical synapsis, & the resistor represents an electrical synapsis. Adapted from Schneider-2018
  • Figure 4: Illustration of the Lateral Giant Escape reflex Microcircuits within the left hemiganglion of segment A2, counterparts of which are repeated in segments A1 through A5. Adapted from Vu-1997
  • Figure 5: The FeedForward Excitation (FFE) Motif.
  • ...and 25 more figures