CoCap: Coordinated motion Capture for multi-actor scenes in outdoor environments

TL;DR

CoCap addresses outdoor multi-actor motion capture by integrating Conflict-Based Search with a perception-focused planning objective to coordinate multiple camera-equipped drones. It introduces a constraint-tree framework that resolves inter-robot conflicts while maximizing multi-view coverage, and couples this with a fast single-agent view search for real-time operation. The approach is evaluated in corridor and bottleneck scenes, showing that coordinated planning outperforms greedy sequential planning and approaches unconstrained planning, with the single-agent search offering substantial runtime advantages. These results indicate strong potential for practical, occlusion-aware, multi-UAV motion capture in cluttered outdoor environments.

Abstract

Motion capture has become increasingly important, not only in computer animation but also in emerging fields like the virtual reality, bioinformatics, and humanoid training. Capturing outdoor environments offers extended horizon scenes but introduces challenges with occlusions and obstacles. Recent approaches using multi-drone systems to capture multiple actor scenes often fail to account for multi-view consistency and reasoning across cameras in cluttered environments. Coordinated motion Capture (CoCap), inspired by Conflict-Based Search (CBS), addresses this issue by coordinating view planning to ensure multi-view reasoning during conflicts. In scenarios with high occlusions and obstacles, where the likelihood of inter-robot collisions increases, CoCap demonstrates performance that approaches the ideal outcomes of unconstrained planning, outperforming existing sequential planning methods. Additionally, CoCap offers a single-robot view search approach for real-time applications in dense environments.

Figures (5)

• Figure 1: Coordinated View Planning: Coverage optimization on dynamic actors with flying cameras in an occlusion-aware and obstacle-clustered environment where camera extrinsic positions across robots are negotiated.
• Figure 2: Sequential (Greedy) View Planning: On the left, there is the sequential view planning of multiple camera positions, where there are egocentric behaviors across multiple viewpoints as seen in the three camera outputs on the left under greedy planning. Coordinated planning, on right: we propose a coordinated view planning approach where there is pixel-level negotiation amongst view positions to allow non-egocentric behaviors as seen in the three camera outputs on the right under coordinated planning. In general, we refer to behaviors where agents prefer to optimize their own reward to potential detriment of others as egocentric and behaviors where agents act with respect for mutual constraints non-egocentric.
• Figure 3: Problem representation of the gimbaled camera (also formulated as a robot, ${\mathcal{R}}$) with projection matrix facilitating coverage on dynamic targets with occlusion and obstacles.
• Figure 4: Scenarios requiring high camera coordination due to conflicting trajectories. (a) Corridor scenario with two actors navigating narrow passageways. (b) Bottleneck scenario with four actors moving through a confined region with intersecting paths.
• Figure 5: Scale rewards using multiple cameras performing view planning using No inter-robot Constraint, Sequential Constraint, and Conflict Based MDP Value Iteration, over total planning horizon for the environment.