Quantifying the biomimicry gap in biohybrid robot-fish pairs

TL;DR

This work uses a biomimetic lure of a rummy-nose tetra fish and a neural network model for generating biomimetic social interactions and demonstrates that this biohybrid system generates social interactions mirroring those of genuine fish pairs.

Abstract

Biohybrid systems in which robotic lures interact with animals have become compelling tools for probing and identifying the mechanisms underlying collective animal behavior. One key challenge lies in the transfer of social interaction models from simulations to reality, using robotics to validate the modeling hypotheses. This challenge arises in bridging what we term the "biomimicry gap", which is caused by imperfect robotic replicas, communication cues and physics constraints not incorporated in the simulations, that may elicit unrealistic behavioral responses in animals. In this work, we used a biomimetic lure of a rummy-nose tetra fish (Hemigrammus rhodostomus) and a neural network (NN) model for generating biomimetic social interactions. Through experiments with a biohybrid pair comprising a fish and the robotic lure, a pair of real fish, and simulations of pairs of fish, we demonstrate that our biohybrid system generates social interactions mirroring those of genuine fish pairs. Our analyses highlight that: 1) the lure and NN maintain minimal deviation in real-world interactions compared to simulations and fish-only experiments, 2) our NN controls the robot efficiently in real-time, and 3) a comprehensive validation is crucial to bridge the biomimicry gap, ensuring realistic biohybrid systems.

Figures (9)

• Figure 1: Illustration of the sources of the biomimicry gap. (1) The modeling phase may introduce a first source of discrepancy between the effect of social interactions on the swimming patterns in the model and the ones observed in real fish. (2) A second source of discrepancy between the visual appearance of the lure and that of a real fish might introduce imperfect communication cues and elicit unrealistic behavioral responses from neighboring organisms. (3) Finally, a third source of discrepancy between the characteristics of the movement produced by the model and its realization by the lure occurs when the numerical model is transferred to real-world scenarios due to the physics constrains that were not accounted for in the model. H. rhodostomus photo was taken by David Villa ScienceImage/CBI/CNRS, Toulouse.
• Figure 2: Closed-loop robot control with Deep Learning Interaction (DLI) model. (a) We use the top setup camera to track all agents (fish and/or lure) in real-time, and store unique trajectories for each agent. A $5 \times 11$ vector of individual and collective states, spanning 5 timesteps is fed to the DLI. (b) The DLI outputs two acceleration distributions, one for each Cartesian component. Then, the acceleration is used to compute the updated desired speed and position for $t+1$, which are communicated to the robot.
• Figure 3: Individual and collective variables. For the focal individual $i$ (light gray), we define the individual quantities: $\vec{u}_i$, its Cartesian position; $\vec{v}_i$, its instantaneous velocity; $r_{\rm w}^i$, its distance to the wall; $\phi_{i}$, its heading angle. We also define the collective quantities from $i$'s perspective when another individual $j$ (dark gray) is also present in the circular tank of radius $R=25$ cm: $d_{ij}$, the interindividual distance; $\phi_{ij}$, the heading difference between both individuals; $\psi_{ij}$, the angle with which individual $j$ is perceived by the focal individual $i$. Note that, for visualization purposes, the size of agents is not to scale.
• Figure 4: Instantaneous individual quantities.(a) Speed $V$ probability density function. (b) Distance to the wall $r_{\rm w}$ probability density function. (c) Angle of incidence to the wall $\theta_{\rm w}$ probability density function. Dark gray, blue, and red colors correspond to the distributions of the fish-only experiment, the DLI simulated pairs, and the DLI biohybrid pairs, respectively. In all PDFs, the colored dot corresponds to the median, and the thick horizontal black line corresponds to the limits of the first and third quartile. The top inset plots depict the PDFs of the DLI biohybrid pair experiments, where the dotted and dashed lines correspond to the robot's and its neighbor's distributions, respectively. The shaded areas correspond to the standard deviation.
• Figure 5: Instantaneous collective quantities.(a) Interindividual distance $d_{ij}$ probability density function. (b) Difference in heading angles $|\phi_{ij}|$ probability density function. (c) Viewing angle $\psi_{ij}$ probability density function. Dark gray, blue, and red colors correspond to the distributions of the experiment, DLI simulated pairs and DLI biohybrid pairs, respectively. In all PDFs, the colored dot corresponds to the median, and the thick horizontal black line corresponds to the limits of the first and third quartile. The inset plots depict the PDFs of the DLI biohybrid pair experiments where the dotted, dashed, and solid lines correspond to the robot, neighbor and average agent distributions, respectively. The shaded areas correspond to the standard deviation.