Control Your Robot: A Unified System for Robot Control and Policy Deployment

TL;DR

The paper tackles cross-platform fragmentation in robot control for embodied intelligence by introducing Control Your Robot, a modular, unified framework that links robot registration, control, and a data-to-deployment pipeline. It demonstrates a cohesive open-source system with standardized APIs that support low-latency data collection, multi-modal data handling, and end-to-end policy deployment across diverse hardware. Empirical results from single-arm and dual-arm experiments show high fidelity to expert trajectories and effective policy learning via imitation learning (ACT) and vision-language-action approaches, while also highlighting generalization challenges under stochastic conditions. The work offers a scalable, reproducible approach to cross-platform robotic learning and emphasizes open-source availability to accelerate adoption and extension across platforms and policies.

Abstract

Cross-platform robot control remains difficult because hardware interfaces, data formats, and control paradigms vary widely, which fragments toolchains and slows deployment. To address this, we present Control Your Robot, a modular, general-purpose framework that unifies data collection and policy deployment across diverse platforms. The system reduces fragmentation through a standardized workflow with modular design, unified APIs, and a closed-loop architecture. It supports flexible robot registration, dual-mode control with teleoperation and trajectory playback, and seamless integration from multimodal data acquisition to inference. Experiments on single-arm and dual-arm systems show efficient, low-latency data collection and effective support for policy learning with imitation learning and vision-language-action models. Policies trained on data gathered by Control Your Robot match expert demonstrations closely, indicating that the framework enables scalable and reproducible robot learning across platforms.

Figures (2)

• Figure 1: System Design. "Control Your Robot" provides a unified workflow that integrates robot control through controller and sensor registration. It enables seamless data collection, model training, and deployment on real hardware with diverse teleoperation devices. The system also includes data-analysis plugins (rapid visualization and efficient offline evaluation) for continuous assessment and improvement.
• Figure 2: Left: We assess distinct manipulation tasks: Place Can on Pot (place a can on a pot edge); Pick Dual Bottles (simultaneously grasp two irregular bottles); Put Cup in Cabinet (handle recognition, drawer operation, placement, and closure); Stack Two Bowls (sequential pick-and-stack).Right: The section depicts the mean difference curves of action trajectories from the ACT model, obtained by replaying the trained model on the same tasks, aiming to evaluate its prediction accuracy.