Cooperation Is All You Need

TL;DR

Cooperator is inspired by recent neurobiological breakthroughs suggesting that the cellular foundations of mental life depend on context-sensitive pyramidal neurons in the neocortex which have two functionally distinct points.

Abstract

Going beyond 'dendritic democracy', we introduce a 'democracy of local processors', termed Cooperator. Here we compare their capabilities when used in permutation invariant neural networks for reinforcement learning (RL), with machine learning algorithms based on Transformers, such as ChatGPT. Transformers are based on the long standing conception of integrate-and-fire 'point' neurons, whereas Cooperator is inspired by recent neurobiological breakthroughs suggesting that the cellular foundations of mental life depend on context-sensitive pyramidal neurons in the neocortex which have two functionally distinct points. Weshow that when used for RL, an algorithm based on Cooperator learns far quicker than that based on Transformer, even while having the same number of parameters.

Figures (3)

• Figure 1: Permutation invariant RL agent (PyBullet Ant) adapting to sensory substitutions: Cooperator vs Transformer vaswani2017attentiontang2021sensory. In a fair comparison, with the same number of parameters, Cooperator learns far quicker than Transformer. See demo: will disclose after double-blind review.
• Figure 2: (A): Point neuron-based Transformer. Scaled Dot-Product Attention or Multi-Head Attention vaswani2017attention used to model permutation invariant RL agent tang2021sensory. (B) A simple representation of Point neuron-based Transformer for permutation invariant PyBullet Ant RL agent. The point neurons simply sum up all the inputs with an assumption that they have the same chance of affecting the neuron's output. (C) Context-sensitive neuron-based Cooperator used to model permutation invariant RL. (D) Functional depiction of a context-sensitive neuron with two points of integration whose contextual integration zone receives proximal context (P) from neighboring sensory 1 neurons, distal context (D) from more distant parts of the network (sensory neurons 2-N), and universal context (U) representing Q. The integrated context (C) is used as an average opinion of the neighboring neurons to decide whether to transmit the information or not. Higher the value of C, higher the probability of transmitting the information. For more details, see adeel2022contextadeel2020conscious.
• Figure 3: Training Results: In both Cart-Pole and PyBullet problems, Cooperator with the same architecture and number of parameters, learns far quicker than Transformer and previously proposed neuro-modulatory functions. In tang2021sensory, the authors only presented testing results, here we present both training and testing results. See demo: will disclose after double-blind review.