FlashBack: Consistency Model-Accelerated Shared Autonomy

TL;DR

The paper addresses the challenge of real-time, goal-agnostic shared autonomy in robotics. It introduces Consistency Shared Autonomy (CSA), a framework that distills a diffusion-based teacher into a one-step denoiser via consistency modeling, enabling microsecond-scale action generation. By training a teacher diffusion model (EDM) and distilling its flow into a student CM, CSA achieves fast, high-fidelity assistance without explicit goal prediction and with minimal training data. The approach demonstrates strong performance across low- to high-precision tasks, including real-world peg-insertion, and shows robustness to user imperfections through surrogate pilots, highlighting practical applicability for real-time robotic collaboration.

Abstract

Shared autonomy is an enabling technology that provides users with control authority over robots that would otherwise be difficult if not impossible to directly control. Yet, standard methods make assumptions that limit their adoption in practice-for example, prior knowledge of the user's goals or the objective (i.e., reward) function that they wish to optimize, knowledge of the user's policy, or query-level access to the user during training. Diffusion-based approaches to shared autonomy do not make such assumptions and instead only require access to demonstrations of desired behaviors, while allowing the user to maintain control authority. However, these advantages have come at the expense of high computational complexity, which has made real-time shared autonomy all but impossible. To overcome this limitation, we propose Consistency Shared Autonomy (CSA), a shared autonomy framework that employs a consistency model-based formulation of diffusion. Key to CSA is that it employs the distilled probability flow of ordinary differential equations (PF ODE) to generate high-fidelity samples in a single step. This results in inference speeds significantly than what is possible with previous diffusion-based approaches to shared autonomy, enabling real-time assistance in complex domains with only a single function evaluation. Further, by intervening on flawed actions at intermediate states of the PF ODE, CSA enables varying levels of assistance. We evaluate CSA on a variety of challenging simulated and real-world robot control problems, demonstrating significant improvements over state-of-the-art methods both in terms of task performance and computational efficiency.

Figures (11)

• Figure 1: A visualization of the result of using our proposed Consistency Shared Autonomy (CSA) algorithm in comparison to the state-of-the-art DDPM-based shared autonomy baseline on three challenging control tasks.
• Figure 2: Distillation of PF ODE flow: Select two distinct states $\{a^t,a^{t-1}\}$ along the same trajectory, the CM enforces that predictions converge to the same target $\hat{a}^0$.
• Figure 3: Training Process of EDM (teacher) model.
• Figure 4: Training Process of CSA (student) model.
• Figure 5: Environment Setting