Human-guided Swarms: Impedance Control-inspired Influence in Virtual Reality Environments

TL;DR

This work tackles the problem of scalable, fine-grained human supervision for swarms by introducing an impedance-control-inspired influence mechanism that blends macroscopic human input with autonomous Couzin-based swarm dynamics. Implemented in a VR environment (HTC Vive, Unreal Engine 4, AirSim) with a 16-drone swarm, the approach represents human input as an additive term $d_i'(t+\tau) = d_i(t+\tau) + \alpha u_i(t)$, where the influence $u_i(t)$ is formed from a stiffness-damping map and a plane-normal mapping to the VR controller plane. The key contributions include the formalization of a diagonal $K$ and $B$–based influence kernel, an explicit mapping from controller motion to swarm direction, and an empirical demonstration that nonzero influence ($\alpha>0$) allows the swarm to traverse narrow canyons while preserving emergent behavior. The results suggest that such blended control supports mission objectives requiring macroscopic guidance without eroding autonomous swarm dynamics, offering a practical pathway for rapid iteration and testing in VR before deployment on real swarms.

Abstract

Prior works in human-swarm interaction (HSI) have sought to guide swarm behavior towards established objectives, but may be unable to handle specific scenarios that require finer human supervision, variable autonomy, or application to large-scale swarms. In this paper, we present an approach that enables human supervisors to tune the level of swarm control, and guide a large swarm using an assistive control mechanism that does not significantly restrict emergent swarm behaviors. We develop this approach in a virtual reality (VR) environment, using the HTC Vive and Unreal Engine 4 with AirSim plugin. The novel combination of an impedance control-inspired influence mechanism and a VR test bed enables and facilitates the rapid design and test iterations to examine trade-offs between swarming behavior and macroscopic-scale human influence, while circumventing flight duration limitations associated with battery-powered small unmanned aerial system (sUAS) systems. The impedance control-inspired mechanism was tested by a human supervisor to guide a virtual swarm consisting of 16 sUAS agents. Each test involved moving the swarm's center of mass through narrow canyons, which were not feasible for a swarm to traverse autonomously. Results demonstrate that integration of the influence mechanism enabled the successful manipulation of the macro-scale behavior of the swarm towards task completion, while maintaining the innate swarming behavior.

Figures (7)

• Figure 1: Feedback loop depicting flow of information across controllers and virtual reality (VR) environment. Human supervisor can provide continuous macroscopic influence to the sUAS swarm in a form of blended or shared control, if innate system behaviors fails to meet mission objectives.
• Figure 2: Impedance control-inspired influence mechanism relies on the distance of the sUAS position $\textbf{x}_i$ from the $XY$ plane (defined by the unit normal vector $\hat{\textbf{n}}$ of the VR controller) to evaluate control effort. Only one VR controller is shown in this schematic, though both are used to determine the final influence effect on the swarming agents.
• Figure 3: Human supervisor exerts macroscopic influence on autonomous swarm using hand-held VR controllers to enable it to successfully traverse a narrow canyon (visible on right monitor screen). Our virtual reality setup uses HTC Vive for control and headset to display the scene to the human supervisor, and Lighthouse for positional tracking (not pictured).
• Figure 4: Algorithmic application of pulse input-like influence along a single axis and its effect on the the swarm's movement along that axis. Programatically-applied inputs demonstrate the intended effects of the influence mechanism in a scenario without explicit human influence $(\alpha = 5)$.
• Figure 5: Agent positions and swarm mean position: (a) with autonomous operation in cohesive flight $(\alpha = 0)$, and (b) guided by human supervisor during milling (with gain $\alpha=5$). Grey vertical planes represent walls, with the narrow gap representing a canyon. Dark gray line denotes projection of mean swarm position onto the $XY$ ground plane.