Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Of Mice and Machines: A Comparison of Learning Between Real World Mice and RL Agents

Shuo Han, German Espinosa, Junda Huang, Daniel A. Dombeck, Malcolm A. MacIver, Bradly C. Stadie

TL;DR

The paper compares learning in real mice and RL agents within a predator-avoidance task to identify gaps in risk assessment. It shows that standard RL often deprioritizes self-preservation, motivating two biologically inspired mechanisms—Trauma-Inspired Safety Buffer and variance-penalized TD learning—to induce more mouse-like risk aversion. Empirical results demonstrate substantial gains in biological plausibility, with high visitation overlap and increased waiting behavior, and an LLM agent reveals broader AI alignment challenges. These findings highlight practical paths toward aligning artificial agents with natural risk-sensitive decision-making in high-stakes environments.

Abstract

Recent advances in reinforcement learning (RL) have demonstrated impressive capabilities in complex decision-making tasks. This progress raises a natural question: how do these artificial systems compare to biological agents, which have been shaped by millions of years of evolution? To help answer this question, we undertake a comparative study of biological mice and RL agents in a predator-avoidance maze environment. Through this analysis, we identify a striking disparity: RL agents consistently demonstrate a lack of self-preservation instinct, readily risking ``death'' for marginal efficiency gains. These risk-taking strategies are in contrast to biological agents, which exhibit sophisticated risk-assessment and avoidance behaviors. Towards bridging this gap between the biological and artificial, we propose two novel mechanisms that encourage more naturalistic risk-avoidance behaviors in RL agents. Our approach leads to the emergence of naturalistic behaviors, including strategic environment assessment, cautious path planning, and predator avoidance patterns that closely mirror those observed in biological systems.

Of Mice and Machines: A Comparison of Learning Between Real World Mice and RL Agents

TL;DR

The paper compares learning in real mice and RL agents within a predator-avoidance task to identify gaps in risk assessment. It shows that standard RL often deprioritizes self-preservation, motivating two biologically inspired mechanisms—Trauma-Inspired Safety Buffer and variance-penalized TD learning—to induce more mouse-like risk aversion. Empirical results demonstrate substantial gains in biological plausibility, with high visitation overlap and increased waiting behavior, and an LLM agent reveals broader AI alignment challenges. These findings highlight practical paths toward aligning artificial agents with natural risk-sensitive decision-making in high-stakes environments.

Abstract

Recent advances in reinforcement learning (RL) have demonstrated impressive capabilities in complex decision-making tasks. This progress raises a natural question: how do these artificial systems compare to biological agents, which have been shaped by millions of years of evolution? To help answer this question, we undertake a comparative study of biological mice and RL agents in a predator-avoidance maze environment. Through this analysis, we identify a striking disparity: RL agents consistently demonstrate a lack of self-preservation instinct, readily risking ``death'' for marginal efficiency gains. These risk-taking strategies are in contrast to biological agents, which exhibit sophisticated risk-assessment and avoidance behaviors. Towards bridging this gap between the biological and artificial, we propose two novel mechanisms that encourage more naturalistic risk-avoidance behaviors in RL agents. Our approach leads to the emergence of naturalistic behaviors, including strategic environment assessment, cautious path planning, and predator avoidance patterns that closely mirror those observed in biological systems.
Paper Structure (26 sections, 12 equations, 17 figures, 1 table, 1 algorithm)

This paper contains 26 sections, 12 equations, 17 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Experiment Setup
  3. Reinforcement Learning (RL)
  4. Online RL: Learning through Interaction
  5. Model-Free Methods
  6. Model-Based Methods
  7. Comparison Between Mouse Behavior and RL Agent Behavior
  8. Changing RL Behaviors to Match Biology
  9. Trauma-Inspired Safety Memory in RL Agents
  10. Learning to Wait Through Variance-Penalized Temporal Difference (TD) Learning
  11. Challenges in Closing the Mice and Machines Gap
  12. What about LLM agents?
  13. Related Work
  14. Discussion
  15. Environment Details
  16. ...and 11 more sections

Figures (17)

  • Figure 1: Real Mouse Setting
  • Figure 2: RL Setting: Mouse's view (left), predator's view (right).
  • Figure 3: Trajectory plots: RL (left) vs Mouse (right). Blue indicates wall-following trajectories (thigmotaxis), while red indicates non-wall-following trajectories.
  • Figure 4: Density plot of RL (left) vs Mouse (right)
  • Figure 5: An example of distinct behaviors: Mouse (left) exhibits hesitation (dense segments near entrance), while RL agent (right) takes a direct path. Additional examples are in the Appendix.
  • ...and 12 more figures