Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Controlling Fish Schools via Reinforcement Learning of Virtual Fish Movement

Yusuke Nishii, Hiroaki Kawashima

Abstract

This study investigates a method to guide and control fish schools using virtual fish trained with reinforcement learning. We utilize 2D virtual fish displayed on a screen to overcome technical challenges such as durability and movement constraints inherent in physical robotic agents. To address the lack of detailed behavioral models for real fish, we adopt a model-free reinforcement learning approach. First, simulation results show that reinforcement learning can acquire effective movement policies even when simulated real fish frequently ignore the virtual stimulus. Second, real-world experiments with live fish confirm that the learned policy successfully guides fish schools toward specified target directions. Statistical analysis reveals that the proposed method significantly outperforms baseline conditions, including the absence of stimulus and a heuristic "stay-at-edge" strategy. This study provides an early demonstration of how reinforcement learning can be used to influence collective animal behavior through artificial agents.

Controlling Fish Schools via Reinforcement Learning of Virtual Fish Movement

Abstract

This study investigates a method to guide and control fish schools using virtual fish trained with reinforcement learning. We utilize 2D virtual fish displayed on a screen to overcome technical challenges such as durability and movement constraints inherent in physical robotic agents. To address the lack of detailed behavioral models for real fish, we adopt a model-free reinforcement learning approach. First, simulation results show that reinforcement learning can acquire effective movement policies even when simulated real fish frequently ignore the virtual stimulus. Second, real-world experiments with live fish confirm that the learned policy successfully guides fish schools toward specified target directions. Statistical analysis reveals that the proposed method significantly outperforms baseline conditions, including the absence of stimulus and a heuristic "stay-at-edge" strategy. This study provides an early demonstration of how reinforcement learning can be used to influence collective animal behavior through artificial agents.
Paper Structure (21 sections, 18 equations, 17 figures, 4 tables)

This paper contains 21 sections, 18 equations, 17 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Research Background
  3. Research Objectives
  4. Related Work
  5. Structure of the Report
  6. Interaction System for Fish Schools and Virtual Fish
  7. Problem Formulation
  8. Rationale for Using Virtual Fish
  9. Camera-Based Measurement of Fish Schools
  10. Reinforcement Learning of Virtual Fish Movement
  11. Markov Decision Process for Fish School Control
  12. Reinforcement Learning via Q-Learning
  13. Preliminary Evaluation in a Simulation Environment
  14. Modeling the Movement of Real Fish
  15. Evaluation of Robustness Against Stochastic Behavior
  16. ...and 6 more sections

Figures (17)

  • Figure 1: Overview of the interaction system between a fish school and virtual fish.
  • Figure 2: Image and arrangement of the virtual fish, including the background color.
  • Figure 3: Example of a captured image. Due to the light from the display, the real fish appear dark as silhouettes.
  • Figure 4: Interaction between the agent and the environment in reinforcement learning.
  • Figure 5: Definition of cells. When the number of divisions is $W$, the cell at the target end is defined as $W-1$, and the cell at the opposite end is defined as $0$.
  • ...and 12 more figures