A Latency-Aware Framework for Visuomotor Policy Learning on Industrial Robots

TL;DR

A latency-aware execution strategy is introduced that schedules finite-horizon, policy-predicted action sequences based on temporal feasibility, enabling asynchronous inference and execution without modifying policy architectures or training and highlights the importance of explicitly handling latency for reliable closed-loop deployment of visuomotor policies on industrial robots.

Abstract

Industrial robots are increasingly deployed in contact-rich construction and manufacturing tasks that involve uncertainty and long-horizon execution. While learning-based visuomotor policies offer a promising alternative to open-loop control, their deployment on industrial platforms is challenged by a large observation-execution gap caused by sensing, inference, and control latency. This gap is significantly greater than on low-latency research robots due to high-level interfaces and slower closed-loop dynamics, making execution timing a critical system-level issue. This paper presents a latency-aware framework for deploying and evaluating visuomotor policies on industrial robotic arms under realistic timing constraints. The framework integrates calibrated multimodal sensing, temporally consistent synchronization, a unified communication pipeline, and a teleoperation interface for demonstration collection. Within this framework, we introduce a latency-aware execution strategy that schedules finite-horizon, policy-predicted action sequences based on temporal feasibility, enabling asynchronous inference and execution without modifying policy architectures or training. We evaluate the framework on a contact-rich industrial assembly task while systematically varying inference latency. Using identical policies and sensing pipelines, we compare latency-aware execution with blocking and naive asynchronous baselines. Results show that latency-aware execution maintains smooth motion, compliant contact behavior, and consistent task progression across a wide range of latencies while reducing idle time and avoiding instability observed in baseline methods. These findings highlight the importance of explicitly handling latency for reliable closed-loop deployment of visuomotor policies on industrial robots.

Figures (15)

• Figure 1: The observation--execution gap with non-negligible latency sources: observation, inference, and execution (see Section \ref{['latency_sources']} for details).
• Figure 2: Policy interface.
• Figure 3: Experimental setup with two six-axis industrial robotic arms and a VR system used for teleoperation-based data collection.
• Figure 4: End effector with integrated force/torque sensor, anti-collision sensor, and eye-in-hand camera.
• Figure 5: Hardware and software communication stack used in the experimental platform.