Visual collective behaviors on spherical robots

TL;DR

This work addresses vision-based collective motion by implementing a robot-in-the-loop visual flocking model on 10 spherical Sphero robots, using only early-vision cues (angular position, angular size, optic flow) augmented by a visual anchor to confine the flock. The authors formulate per-robot angular velocity components from retinal cues, introduce an avoidance term, and rely on an external panorama reconstructed for each robot, enabling independent, parallel control without inter-robot communication. The key contributions include the visual anchor mechanism, a robust simulation-to-robot validation showing swarming, milling, and bistable phases, and phase-diagram alignment between simulation and hardware, indicating a minimal yet effective visual model. This approach bridges the gap between numerical flocking models and physical experiments, offering a scalable, sensor-grounded framework for visual collective behaviors in robotics with practical implications for swarm robotics and embodied AI.

Abstract

The implementation of collective motion, traditionally, disregard the limited sensing capabilities of an individual, to instead assuming an omniscient perception of the environment. This study implements a visual flocking model in a ``robot-in-the-loop'' approach to reproduce these behaviors with a flock composed of 10 independent spherical robots. The model achieves robotic collective motion by only using panoramic visual information of each robot, such as retinal position, optical size and optic flow of the neighboring robots. We introduce a virtual anchor to confine the collective robotic movements so to avoid wall interactions. For the first time, a simple visual robot-in-the-loop approach succeed in reproducing several collective motion phases, in particular, swarming, and milling. Another milestone achieved with by this model is bridging the gap between simulation and physical experiments by demonstrating nearly identical behaviors in both environments with the same visual model. To conclude, we show that our minimal visual collective motion model is sufficient to recreate most collective behaviors on a robot-in-the-loop system that is scalable, behaves as numerical simulations predict and is easily comparable to traditional models.

Figures (9)

• Figure 1: Modeling collective behaviors from optic flow and retinal cues castro2024: a) View of the flock positions. b) Representation of flock positions after recognition at time step $t$. Each $i$-particle is described by its XY-coordinates and heading unit vector $\mathbf{e}_i$, with heading angle $\theta_i$. c) Point-of-view of particle 0, showing the relative retinal positions of each particle. The arrows represent the perceived optic flow, with the white arrow indicating the apparent velocity ($\mathbf{V}_{i,k}$). Only the azimuthal component of $\mathbf{V}_{i,k}$ is shown for retinal object $k$ as perceived by $i=0$. d) $\mathcal{V}(\phi)$: Binary panorama representation of the field of view of particle 0. e) Apparent distance $\mathcal{R}_i(\phi)$ for particle 0. f) Optic flow $\mathcal{O}_i(\phi)$ perceived by particle 1. g) Optic flow divergence $\mathcal{D}_i(\phi)$ perceived by particle 1.
• Figure 2: Spherical robotic platform. a) Sphero Bolt® wireless educational robots. b) Example of the robots' movement, c) Heading control by differential rotation on the two opposite inner wheels, d) Induced nose-up pitch during forward displacement.
• Figure 3: Experimental Setup and external infrastructure. a) Photograph of the experimental setup. b) Sample image captured by the camera (Daheng Imaging MER2-230-168U3C camera equipped with a 4 mm C-mount lens). c) Whole-robot body image segmentation: Segmentation of the robot's body based on the captured image. d) Recognition of individual light sources and heading: Identification of the front and rear LEDs allows calculation of the robot's heading vector. e) Robot identification and virtualization: Assignment of a unique identifier to each robot and visualization of their positions and orientations.
• Figure 4: Binary Panorama Reconstruction. a) 2D virtual representation of the binary panorama: The red goal indicates the desired location of the flock. b) Mixed-reality perspective from particle 0: Displays the omniscient red goal as observed from the particle's point of view. c) Graph representation of all $\mathcal{V}(\phi,t)$: $\mathcal{V}(\phi,t)$ represents the binary panorama view of the field of vision for particle i, with the anchor displayed as a distinct channel (red). This graph can be efficiently stored as a tuple list, representing the rising and falling edges of the binary panorama.
• Figure 5: Functional diagram of the collective motion algorithm with robots-in-the-loop.