Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolution imposes an inductive bias that alters and accelerates learning dynamics

Benjamin Midler, Alejandro Pan Vazquez

TL;DR

This work addresses why brains learn rapidly with limited data by exploring how evolutionary optimization can shape online learning. The authors implement Evolutionary Conditioning (EC), which partnerships a genetic algorithm for generational selection with gradient-descent fine-tuning within each generation, keeping the two processes distinct. EC yields latent learning dynamics across reinforcement and supervised tasks and dramatically speeds up subsequent fine-tuning on a semantic cognition task. The results imply that evolution provides an inductive bias that tunes learning trajectories rather than simply encoding prior task performance. Future work should extend EC to additional tasks, evolve architectures and physiology, and assess how multi-level brain structure interacts with learning.

Abstract

The learning dynamics of biological brains and artificial neural networks are of interest to both neuroscience and machine learning. A key difference between them is that neural networks are often trained from a randomly initialized state whereas each brain is the product of generations of evolutionary optimization, yielding innate structures that enable few-shot learning and inbuilt reflexes. Artificial neural networks, by contrast, require non-ethological quantities of training data to attain comparable performance. To investigate the effect of evolutionary optimization on the learning dynamics of neural networks, we combined algorithms simulating natural selection and online learning to produce a method for evolutionarily conditioning artificial neural networks, and applied it to both reinforcement and supervised learning contexts. We found the evolutionary conditioning algorithm, by itself, performs comparably to an unoptimized baseline. However, evolutionarily conditioned networks show signs of unique and latent learning dynamics, and can be rapidly fine-tuned to optimal performance. These results suggest evolution constitutes an inductive bias that tunes neural systems to enable rapid learning.

Evolution imposes an inductive bias that alters and accelerates learning dynamics

TL;DR

This work addresses why brains learn rapidly with limited data by exploring how evolutionary optimization can shape online learning. The authors implement Evolutionary Conditioning (EC), which partnerships a genetic algorithm for generational selection with gradient-descent fine-tuning within each generation, keeping the two processes distinct. EC yields latent learning dynamics across reinforcement and supervised tasks and dramatically speeds up subsequent fine-tuning on a semantic cognition task. The results imply that evolution provides an inductive bias that tunes learning trajectories rather than simply encoding prior task performance. Future work should extend EC to additional tasks, evolve architectures and physiology, and assess how multi-level brain structure interacts with learning.

Abstract

The learning dynamics of biological brains and artificial neural networks are of interest to both neuroscience and machine learning. A key difference between them is that neural networks are often trained from a randomly initialized state whereas each brain is the product of generations of evolutionary optimization, yielding innate structures that enable few-shot learning and inbuilt reflexes. Artificial neural networks, by contrast, require non-ethological quantities of training data to attain comparable performance. To investigate the effect of evolutionary optimization on the learning dynamics of neural networks, we combined algorithms simulating natural selection and online learning to produce a method for evolutionarily conditioning artificial neural networks, and applied it to both reinforcement and supervised learning contexts. We found the evolutionary conditioning algorithm, by itself, performs comparably to an unoptimized baseline. However, evolutionarily conditioned networks show signs of unique and latent learning dynamics, and can be rapidly fine-tuned to optimal performance. These results suggest evolution constitutes an inductive bias that tunes neural systems to enable rapid learning.
Paper Structure (27 sections, 4 figures)

This paper contains 27 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Results
  3. Evolutionary conditioning implementation
  4. Reinforcement learning reveals latent behavioral structure
  5. Evolutionary conditioning dynamics are distinct from learning dynamics
  6. Evolutionary conditioning facilitates rapid learning
  7. Evolutionary conditioning alters gradient descent learning dynamics
  8. Discussion
  9. Technical Appendices and Supplementary Material
  10. Reinforcement learning: task
  11. Reinforcement learning: gradient descent implementation
  12. Reinforcement learning: genetic algorithm implementation
  13. Reinforcement learning: evolutionary conditioning implementation
  14. Reinforcement learning: analyses and statistics
  15. Model performance comparison
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Brains are optimized by natural selection and online learning. (A) Brains and artificial neural networks engage in online learning by producing behaviors and receiving environmental feedback to guide learning. (B) The fitness of biological brains is optimized over evolutionary timescales by natural selection. This constitutes an additional form of optimization on the initial state of a brain, endowing it with inbuilt structures for innate reflexes and enhanced learning potential. In biology, evolutionary optimization and online learning co-exist as evolutionary pressures act upon the genetic information that produces a new brain that then engages in online learning.
  • Figure 2: EC fails to task-optimize but shows signs of latent learning. (A) Schematic of the reinforcement learning problem. A pole is balancing on a platform that can be moved laterally. Networks are trained to move the platform to keep the pole balancing upright. (B) Task performance per model. Each reward is a timestep during which the pole is balanced. Max per episode is 1,000, n=50 episodes (Mann-Whitney U test ***p<1e-18 Bonferroni corrected). (C) Distribution of network actions in response to 10,000 uniformly sampled task states (vectors of cart position, velocity, pole angle, and angular velocity). EC behavior is more similar to the GA even though performance is no different than an untrained network. Note: GA and EC distributions are highly overlapping. (D) Example behavioral congruence matrices between each model pair. (E) Congruence distributions (green) versus shuffled null (gray), n=100 matrices as in D. EC is internally consistent and significantly similar to successfully trained networks, indicating latent learning (Mann-Whitney U test ***p<4e-12 Bonferroni corrected).
  • Figure 3: EC representational dynamics are unique and do not respond to known regularities in training data. (A) The items-to-attributes dataset used in the semantic cognition dataset. (B) The U matrix produced by the singular value decomposition of the dataset. The U matrix shows the loadings of each attribute per mode. (C) The S matrix produced by the singular value decomposition of the dataset. The values of the diagonal, or singular values, correspond to the precedence of the modes. (D) The VT matrix produced by the singular value decomposition of the dataset. It shows the loadings of each item onto the modes, illustrating how the modes correspond to categorical differentiations (e.g. mode 1 splits plants and animals, mode 2 splits birds and fish). (E) Schematic of dataset similarity structure divided into categorical levels. Solid branches are those shown in I-K, dotted branches are not shown but are equivalent to those from the same hierarchical level. (F-H) Loss curves of each model during training. Both SGD and GA task-optimize, but not EC. (I-K) Representational dynamics of each model throughout training. Both SGD and GA show sequential learning dynamics in which superordinate categories are learned before subordinate. EC, however, learns to distinguish all categorical levels concurrently.
  • Figure 4: EC speeds fine-tuning and alters gradient descent representational dynamics. (A) EC loss curve with colored dots indicating network snapshots taken form throughout training. (B) Loss curves of network snapshots from A being trained to criterion. (C) Same as in B but each network snapshot is expressed as epochs of training required to reach criterion. Even though loss during EC is flat or elevated (shown in A), it accelerates fine-tuning by an order of magnitude. (D) Loss curves for SGD and EC networks trained to criterion. (E) Comparing epochs to train to criterion for a network trained via SGD and a network pre-trained with EC then fine-tuned (Mann-Whitney U test Bonferroni corrected ***p<5e-38). (F) Loss curves (top) and representational dynamics (bottom) for network snapshots from throughout EC as they were fine-tuned. Dynamics are unique not only during EC, but during fine-tuning as well.