NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Data

Abstract

The Nymeria Dataset, released in 2024, is a large-scale collection of in-the-wild human activities captured with multiple egocentric wearable devices that are spatially localized and temporally synchronized. It provides body-motion ground truth recorded with a motion-capture suit, device trajectories, semi-dense 3D point clouds, and in-context narrations. In this paper, we upgrade Nymeria and introduce NymeriaPlus. NymeriaPlus features: (1) improved human motion in Momentum Human Rig (MHR) and SMPL formats; (2) dense 3D and 2D bounding box annotations for indoor objects and structural elements; (3) instance-level 3D object reconstructions; and (4) additional modalities e.g., basemap recordings, audio, and wristband videos. By consolidating these complementary modalities and annotations into a single, coherent benchmark, NymeriaPlus strengthens Nymeria into a more powerful in-the-wild egocentric dataset. We expect NymeriaPlus to bridge a key gap in existing egocentric resources and to support a broader range of research, including unique explorations of multimodal learning for embodied AI.

Figures (10)

• Figure 1: Nymeria and NymeriaPlus motion samples. We compare Nymeria results (blue) against the new MHR (yellow) and SMPL (green) ones. Each frame also shows Aria head and wrist trajectories. NymeriaPlus results exhibit more realistic body proportions and better matching of wrist trajectories, resulting in fewer body self-penetrations and improved placement with respect to the 3D scene.
• Figure 2: Boxy 3D OBB annotation tool interface. (Left) The 3D point cloud and a 3D gizmo are used to modify all nine degrees of freedom of the box directly in 3D. (Right) The current 3D box annotation is projected into the RGB image. Annotators can iterate across views to inspect alignment.
• Figure 3: Basemap 3D OBB annotations. We visualize basemap annotations for three venues from NymeriaPlus.
• Figure 4: Basemap 2D OBB annotations. Visible 3D OBBs from three venues are rendered into sample RGB frames.
• Figure 5: Basemap unique 3D objects histogram. We count the number of unique 3D OBBs across the basemaps for each class. Since we transfer these OBBs to the individual scenario recordings, this is the fundamental distribution over observed objects. Note that transferring the OBBs to scenario recordings increases the viewpoint diversity substantially over just the basemap sequences.