Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning

TL;DR

This work tackles how to enable a humanoid with only egocentric vision to perform dexterous, multi-task manipulation in cluttered environments. It introduces Perceptive Dexterous Control (PDC), a vision-based reinforcement learning framework that uses perception-as-interface with visual markers for task specification and a motion prior (PULSE-X) to support dexterous whole-body control, trained end-to-end with PPO. The key contributions include a vision-driven, task-agnostic policy for tabletop and kitchen tasks, the perception-as-interface paradigm that removes reliance on privileged state, and evidence of emergent human-like behaviors such as active search, demonstrated through extensive ablations and analysis across visual modalities, rewards, and multi-task learning. The results show strong generalization to unseen objects and scenes, highlighting PDC’s potential to close the perception-action loop for realistic animation, robotics, and embodied AI systems.

Abstract

Human behavior is fundamentally shaped by visual perception -- our ability to interact with the world depends on actively gathering relevant information and adapting our movements accordingly. Behaviors like searching for objects, reaching, and hand-eye coordination naturally emerge from the structure of our sensory system. Inspired by these principles, we introduce Perceptive Dexterous Control (PDC), a framework for vision-driven dexterous whole-body control with simulated humanoids. PDC operates solely on egocentric vision for task specification, enabling object search, target placement, and skill selection through visual cues, without relying on privileged state information (e.g., 3D object positions and geometries). This perception-as-interface paradigm enables learning a single policy to perform multiple household tasks, including reaching, grasping, placing, and articulated object manipulation. We also show that training from scratch with reinforcement learning can produce emergent behaviors such as active search. These results demonstrate how vision-driven control and complex tasks induce human-like behaviors and can serve as the key ingredients in closing the perception-action loop for animation, robotics, and embodied AI.

Figures (8)

• Figure 1: Perceptive Dexterous Control (PDC) enables a humanoid equipped with egocentric vision to search for, reach, grasp, and manipulate objects in cluttered kitchen scenes. We use visual perception as the sole interface for indicating which hand to use, which object to grasp, where to move, and which drawer to open.
• Figure 2: Kitchens: Our agent is trained in parallel on a large set of (randomly) procedurally generated kitchens. Each generated kitchen is structurally and visually different. Objects are spawned in random locations on the counter, and the agent starts in a random position and orientation within the scene. This diversity encourages the agent to learn general behaviors, such as search, and robust interaction.
• Figure 3: Perception-as-Interface: PDC instructs the policy through visual signals. We overlay the object of interest using a green mask, use a 3D blue marker to indicate target location, and use colored 2D squares (top corners) to inform the agent which hand to use and when to grasp and release.
• Figure 4: We use perception and proprioception as input to the network, processed by a simple CNN-GRU-MLP architecture.
• Figure 5: Tabletop: PDC is instructed directly from visual signals. A visual (top left and/or right) indicates in purple whether the corresponding hand should be prepared for contact. Changing to dark-blue indicates that contact should be made. Instructing the agent to use both hands enables it to transport larger objects. Changing the indicator back to purple instructs the agent to release the object.