Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalisation to unseen topologies: Towards control of biological neural network activity

Laurens Engwegen, Daan Brinks, Wendelin Böhmer

TL;DR

This paper introduces an environment that procedurally generates neuronal networks with different topologies to investigate the generalisation problem of closed-loop control of activity in biological neural networks and demonstrates the capability to generalise control from a limited number of training networks to unseen test networks.

Abstract

Novel imaging and neurostimulation techniques open doors for advancements in closed-loop control of activity in biological neural networks. This would allow for applications in the investigation of activity propagation, and for diagnosis and treatment of pathological behaviour. Due to the partially observable characteristics of activity propagation, through networks in which edges can not be observed, and the dynamic nature of neuronal systems, there is a need for adaptive, generalisable control. In this paper, we introduce an environment that procedurally generates neuronal networks with different topologies to investigate this generalisation problem. Additionally, an existing transformer-based architecture is adjusted to evaluate the generalisation performance of a deep RL agent in the presented partially observable environment. The agent demonstrates the capability to generalise control from a limited number of training networks to unseen test networks.

Generalisation to unseen topologies: Towards control of biological neural network activity

TL;DR

This paper introduces an environment that procedurally generates neuronal networks with different topologies to investigate the generalisation problem of closed-loop control of activity in biological neural networks and demonstrates the capability to generalise control from a limited number of training networks to unseen test networks.

Abstract

Novel imaging and neurostimulation techniques open doors for advancements in closed-loop control of activity in biological neural networks. This would allow for applications in the investigation of activity propagation, and for diagnosis and treatment of pathological behaviour. Due to the partially observable characteristics of activity propagation, through networks in which edges can not be observed, and the dynamic nature of neuronal systems, there is a need for adaptive, generalisable control. In this paper, we introduce an environment that procedurally generates neuronal networks with different topologies to investigate this generalisation problem. Additionally, an existing transformer-based architecture is adjusted to evaluate the generalisation performance of a deep RL agent in the presented partially observable environment. The agent demonstrates the capability to generalise control from a limited number of training networks to unseen test networks.
Paper Structure (22 sections, 4 equations, 5 figures, 2 tables)

This paper contains 22 sections, 4 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Reinforcement Learning and Generalisation
  3. Neuronal activity
  4. Related Work
  5. Neuronal simulation
  6. Closed-loop control of neuronal activity
  7. Multitask learning
  8. Architecture
  9. Method
  10. Environment
  11. States and observations.
  12. Actions.
  13. Transitions.
  14. Procedural context generation.
  15. Rewards.
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Voltage image of neurons grown in vitro. When the field of view and number of neurons increases, it quickly becomes infeasible to study neuronal dynamics in detail.
  • Figure 2: The architectures with which we experiment. Different RNNs (in TDRQN, DDRQN and DMDRQN) share the same parameters, but keep track of different hidden states. FF blocks placed below each other correspond to layers with the same parameters, but applied to different inputs. The concatenation operation is indicated with $\oplus$. The components of the (baseline) architectures that are schematically visualised on the right correspond to those in TDRQN with the same colour.
  • Figure 3: Performance on 100 training (left) and 100 unseen test (right) contexts for TDRQN, DDRQN, MDDRQN, and DRQN. Performance is averaged over 10 seeds and the standard deviation is shown in the shaded area.
  • Figure 4: Performance on unseen test contexts of TDRQN with different amounts of training contexts. Performance is averaged over 10 seeds and the standard deviation is shown in the shaded area.
  • Figure 5: Three, (a), (b), and (c), instances of networks of 8 neurons that are generated as described in this section. The target neuron has a green colour. Black arrows indicate excitatory connections and red edges show inhibitory connections. A circular layout is used for visualisation purposes.