Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning for Control of Evolutionary and Ecological Processes

Bryce Allen Bagley, Navin Khoshnan, Claudia K Petritsch

TL;DR

The paper addresses directing evolution in cellular populations by marrying Evolutionary Game Theory with reinforcement learning in a multicule-encoded, computable framework. It introduces a computational duality that treats ecology and physiology as computations, yielding a quadratic dynamical system and a POMDP formulation amenable to Posterior Sampling Reinforcement Learning (PSRL) with rigorous regret bounds. Key contributions include a complexity bound for eco-evolutionary control under partial prior knowledge, a general treatment of directed evolution, and an explicit link between AI theory and biophysical evolution, supported by model-predictive control and Bayesian quadrature techniques. The work lays a theoretical foundation for data-driven, experiment-informed control of evolving biological systems, with potential applications in cancer therapy optimization and microbial engineering. The framework provides practical pathways for incorporating prior biological data as Bayesian priors while offering provable learning guarantees in complex, high-dimensional dynamics.

Abstract

As Evolutionary Dynamics moves from the realm of theory into application, algorithms are needed to move beyond simple models. Yet few such methods exist in the literature. Ecological and physiological factors are known to be central to evolution in realistic contexts, but accounting for them generally renders problems intractable to existing methods. We introduce a formulation of evolutionary games which accounts for ecology and physiology by modeling both as computations and use this to analyze the problem of directed evolution via methods from Reinforcement Learning. This combination enables us to develop first-of-their-kind results on the algorithmic problem of learning to control an evolving population of cells. We prove a complexity bound on eco-evolutionary control in situations with limited prior knowledge of cellular physiology or ecology, give the first results on the most general version of the mathematical problem of directed evolution, and establish a new link between AI and biology.

Reinforcement Learning for Control of Evolutionary and Ecological Processes

TL;DR

The paper addresses directing evolution in cellular populations by marrying Evolutionary Game Theory with reinforcement learning in a multicule-encoded, computable framework. It introduces a computational duality that treats ecology and physiology as computations, yielding a quadratic dynamical system and a POMDP formulation amenable to Posterior Sampling Reinforcement Learning (PSRL) with rigorous regret bounds. Key contributions include a complexity bound for eco-evolutionary control under partial prior knowledge, a general treatment of directed evolution, and an explicit link between AI theory and biophysical evolution, supported by model-predictive control and Bayesian quadrature techniques. The work lays a theoretical foundation for data-driven, experiment-informed control of evolving biological systems, with potential applications in cancer therapy optimization and microbial engineering. The framework provides practical pathways for incorporating prior biological data as Bayesian priors while offering provable learning guarantees in complex, high-dimensional dynamics.

Abstract

As Evolutionary Dynamics moves from the realm of theory into application, algorithms are needed to move beyond simple models. Yet few such methods exist in the literature. Ecological and physiological factors are known to be central to evolution in realistic contexts, but accounting for them generally renders problems intractable to existing methods. We introduce a formulation of evolutionary games which accounts for ecology and physiology by modeling both as computations and use this to analyze the problem of directed evolution via methods from Reinforcement Learning. This combination enables us to develop first-of-their-kind results on the algorithmic problem of learning to control an evolving population of cells. We prove a complexity bound on eco-evolutionary control in situations with limited prior knowledge of cellular physiology or ecology, give the first results on the most general version of the mathematical problem of directed evolution, and establish a new link between AI and biology.
Paper Structure (27 sections, 111 equations, 1 figure)

This paper contains 27 sections, 111 equations, 1 figure.

Table of Contents

  1. Dynamical System Model for Multicule Evolutionary Dynamics
  2. Non-static Ecology
  3. POMDP for Multicule Evolutionary Models
  4. Parameter Learning
  5. Computational Complexity
  6. Supplemental Information 1: Derivation of the Evolutionary Dynamics Model
  7. System of 1st Order Matrix Differential Equations
  8. Single 2nd Order Matrix Differential Equation
  9. Computational Complexity of Model Evaluation
  10. System Stability
  11. Supplemental Information 2: Model Predictive Control of the Evolutionary Game
  12. Supplemental Information 3: Reward Function Estimation
  13. Franke-Wolfe Bayesian Quadrature
  14. Calculations
  15. Supplemental Information 4: Derivation of Regret Bounds
  16. ...and 12 more sections

Figures (1)

  • Figure 1: A visual summary of the sketch of an experimental protocol, described above. Red arrows denote the loop described in step 5 of the protocol sketch.