Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Flexible Field-Based Policy Learning Framework for Diverse Robotic Systems and Sensors

Jose Gustavo Buenaventura Carreon, Floris Erich, Roman Mykhailyshyn, Tomohiro Motoda, Ryo Hanai, Yukiyasu Domae

TL;DR

The paper tackles cross-robot generalization for visuomotor manipulation by extending GenDP with D³Fields to a UR5+Azure Kinect setup, while preserving compatibility with ALOHA+RealSense. It introduces a modular, hardware-agnostic workcell framework that enables data collection, training, and evaluation across different robot–camera configurations, achieving 80% success on UR5 and 90% on ALOHA after 100 demonstrations. Key contributions include hardware abstraction for multi-platform diffusion-policy learning, synchronized multi-camera acquisition, and an episodic data-collection pipeline that supports rapid transfer across setups. The results demonstrate data-efficient, cross-platform generalization and set the stage for scalable, reproducible real-world cross-robot learning with diffusion-based visuomotor policies.

Abstract

We present a cross robot visuomotor learning framework that integrates diffusion policy based control with 3D semantic scene representations from D3Fields to enable category level generalization in manipulation. Its modular design supports diverse robot camera configurations including UR5 arms with Microsoft Azure Kinect arrays and bimanual manipulators with Intel RealSense sensors through a low latency control stack and intuitive teleoperation. A unified configuration layer enables seamless switching between setups for flexible data collection training and evaluation. In a grasp and lift block task the framework achieved an 80 percent success rate after only 100 demonstration episodes demonstrating robust skill transfer between platforms and sensing modalities. This design paves the way for scalable real world studies in cross robotic generalization.

A Flexible Field-Based Policy Learning Framework for Diverse Robotic Systems and Sensors

TL;DR

The paper tackles cross-robot generalization for visuomotor manipulation by extending GenDP with D³Fields to a UR5+Azure Kinect setup, while preserving compatibility with ALOHA+RealSense. It introduces a modular, hardware-agnostic workcell framework that enables data collection, training, and evaluation across different robot–camera configurations, achieving 80% success on UR5 and 90% on ALOHA after 100 demonstrations. Key contributions include hardware abstraction for multi-platform diffusion-policy learning, synchronized multi-camera acquisition, and an episodic data-collection pipeline that supports rapid transfer across setups. The results demonstrate data-efficient, cross-platform generalization and set the stage for scalable, reproducible real-world cross-robot learning with diffusion-based visuomotor policies.

Abstract

We present a cross robot visuomotor learning framework that integrates diffusion policy based control with 3D semantic scene representations from D3Fields to enable category level generalization in manipulation. Its modular design supports diverse robot camera configurations including UR5 arms with Microsoft Azure Kinect arrays and bimanual manipulators with Intel RealSense sensors through a low latency control stack and intuitive teleoperation. A unified configuration layer enables seamless switching between setups for flexible data collection training and evaluation. In a grasp and lift block task the framework achieved an 80 percent success rate after only 100 demonstration episodes demonstrating robust skill transfer between platforms and sensing modalities. This design paves the way for scalable real world studies in cross robotic generalization.
Paper Structure (11 sections, 4 figures)

This paper contains 11 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. D3Fields
  4. GenDP
  5. Diffusion Policy
  6. Methodology
  7. Implementation
  8. Evaluation
  9. Conclusion and Discussion
  10. Future work
  11. Acknowledgement

Figures (4)

  • Figure 1: Sample setups used in our experiments.
  • Figure 2: Modular workcell concept. (a) A workcell is defined as a combination of a robot, a control interface, and a camera system. (b) Examples of supported workcells, illustrating the flexibility of the framework to integrate different robots, controllers, and sensors.
  • Figure 3: Workflow of the proposed framework. Episodic demonstrations of a manipulation task are recorded, a visuomotor policy is trained using the GenDP pipeline, and the learned policy is evaluated on the chosen hardware setup using the D³Fields feature extraction pipeline..
  • Figure 4: Example of the grasp-and-lift task: the selected robot approaches the target object, grasps it, and lifts it from the table.