Moving Through Clutter: Scaling Data Collection and Benchmarking for 3D Scene-Aware Humanoid Locomotion via Virtual Reality

TL;DR

This work presents Moving Through Clutter (MTC), an opensource Virtual Reality (VR) based data collection and evaluation framework for scene-aware humanoid locomotion in cluttered environments, and compiles a dataset of 348 trajectories across 145 diverse 3D cluttered scenes, providing a foundation for studying geometry-induced adaptation in humanoid locomotion and developing scene-aware planning and control methods.

Abstract

Recent advances in humanoid locomotion have enabled dynamic behaviors such as dancing, martial arts, and parkour, yet these capabilities are predominantly demonstrated in open, flat, and obstacle-free settings. In contrast, real-world environments such as homes, offices, and public spaces, are densely cluttered, three-dimensional, and geometrically constrained, requiring scene-aware whole-body coordination, precise balance control, and reasoning over spatial constraints imposed by furniture and household objects. However, humanoid locomotion in cluttered 3D environments remains underexplored, and no public dataset systematically couples full-body human locomotion with the scene geometry that shapes it. To address this gap, we present Moving Through Clutter (MTC), an opensource Virtual Reality (VR) based data collection and evaluation framework for scene-aware humanoid locomotion in cluttered environments. Our system procedurally generates scenes with controllable clutter levels and captures embodiment-consistent, whole-body human motion through immersive VR navigation, which is then automatically retargeted to a humanoid robot model. We further introduce benchmarks that quantify environment clutter level and locomotion performance, including stability and collision safety. Using this framework, we compile a dataset of 348 trajectories across 145 diverse 3D cluttered scenes. The dataset provides a foundation for studying geometry-induced adaptation in humanoid locomotion and developing scene-aware planning and control methods.

Figures (6)

• Figure 1: We introduce MTC, a dataset and benchmark for humanoid locomotion in cluttered 3D environments. We first procedurally generate cluttered simulation scenes with diverse geometric constraints, then collect immersive, embodiment-scaled locomotion trajectories using VR-based full-body tracking within these environments. The resulting dataset enables benchmarking of scene-aware humanoid locomotion behaviors.
• Figure 2: System overview of MTC. MTC Capturer: immersive VR-based system for collecting embodiment-scaled human locomotion trajectories in cluttered environments. MTC Dataset: a large-scale collection of locomotion trajectories recorded in procedurally generated cluttered scenes. MTC Benchmark: evaluation framework for measuring scene-aware locomotion behaviors relative to normal walking.
• Figure 3: Examples of cluttered environments from two geometric regimes in MTC: structured domestic layouts (left) and debris-style layouts (right).
• Figure 4: Distribution of realised floor-occupancy ratio $c'$ across 145 generated scenes. Individual scene values are shown as jittered strips below the density curves; dashed lines indicate per-regime means.
• Figure 5: Case study of goal-conditioned route diversity in a representative MTC scene. The left panel visualizes the scene floorplan together with the ground-plane projections of pelvis trajectories under four goal configurations. Markers $\bullet$, $\star$, and $\times$ denote start locations, goal positions, and obstacle-avoidance maneuvers, respectively. Insets (a–d) show representative locomotion behaviors observed along the corresponding routes: (a) crouched lateral shuffling, (b) crouched forward shuffling, (c) high-knee lateral step-over, and (d) prone crawling.