VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Play

TL;DR

VolleyBots presents a high-fidelity drone volleyball testbed that unifies mixed competitive-cooperative gameplay, turn-based interactions, and agile 3D motion control. It delivers a curriculum of tasks from single-agent drills to large-scale multi-agent matches and benchmarks a range of RL, MARL, and game-theoretic methods, including a hierarchical policy that achieves 69.5% in 3v3. The study reveals that on-policy methods excel on low-level control yet struggle with tasks requiring integrated motion and strategy, while off-policy methods often lag, motivating hierarchical approaches. The work also demonstrates sim-to-real potential via zero-shot transfer to real drones, underscoring VolleyBots as a practical platform for advancing embodied intelligence in multi-agent robotic systems.

Abstract

Robot sports, characterized by well-defined objectives, explicit rules, and dynamic interactions, present ideal scenarios for demonstrating embodied intelligence. In this paper, we present VolleyBots, a novel robot sports testbed where multiple drones cooperate and compete in the sport of volleyball under physical dynamics. VolleyBots integrates three features within a unified platform: competitive and cooperative gameplay, turn-based interaction structure, and agile 3D maneuvering. These intertwined features yield a complex problem combining motion control and strategic play, with no available expert demonstrations. We provide a comprehensive suite of tasks ranging from single-drone drills to multi-drone cooperative and competitive tasks, accompanied by baseline evaluations of representative reinforcement learning (RL), multi-agent reinforcement learning (MARL) and game-theoretic algorithms. Simulation results show that on-policy RL methods outperform off-policy methods in single-agent tasks, but both approaches struggle in complex tasks that combine motion control and strategic play. We additionally design a hierarchical policy which achieves 69.5% win rate against the strongest baseline in the 3 vs 3 task, demonstrating its potential for tackling the complex interplay between low-level control and high-level strategy. To highlight VolleyBots' sim-to-real potential, we further demonstrate the zero-shot deployment of a policy trained entirely in simulation on real-world drones.

Figures (9)

• Figure 1: Overview of the VolleyBots Testbed. VolleyBots comprises three components: (1) Environment, supported by Isaac Sim, which defines entities, observations, actions, and reward functions; (2) Tasks, including 3 single-agent tasks, 3 multi-agent cooperative tasks, and 3 multi-agent competitive tasks; and (3) Algorithms, encompassing RL, MARL, and game-theoretic algorithms.
• Figure 2: Proposed tasks in the VolleyBots testbed, inspired by the process of human learning in volleyball. Single-agent tasks evaluate low-level control, while multi-agent cooperative and competitive tasks integrate high-level decision-making with low-level control.
• Figure 3: Cross-play heatmap of the 1 vs 1 and 3 vs 3 competitive tasks.
• Figure 4: Demonstration of the hierarchical policy selecting Serve and Attack drills in the 3 vs 3 task.
• Figure 5: Zero-shot sim-to-real experiment on the Solo Bump task.