Beyond Success: Refining Elegant Robot Manipulation from Mixed-Quality Data via Just-in-Time Intervention

TL;DR

Imitation-learned robotic policies trained on mixed-quality demonstrations exhibit variable execution quality. The authors propose LIBERO-Elegant, an elegance-focused benchmark, and a decoupled refinement framework comprising an Elegance Critic trained via Calibrated Q-Learning and a Just-in-Time Intervention mechanism that selectively refines actions at decision-critical moments without retraining the base policy. Offline training on an Elegance-Enriched Dataset enables calibrated value estimation of action elegance, while JITI uses critic confidence to trigger high-cost, multi-sample refinement only when necessary. Results across simulation and real-world tasks show significant gains in Elegant Success Rate and strong generalization to unseen objects and contexts, highlighting the practical impact of optimizing not just task success but motion quality.

Abstract

Vision-Language-Action (VLA) models have enabled notable progress in general-purpose robotic manipulation, yet their learned policies often exhibit variable execution quality. We attribute this variability to the mixed-quality nature of human demonstrations, where the implicit principles that govern how actions should be carried out are only partially satisfied. To address this challenge, we introduce the LIBERO-Elegant benchmark with explicit criteria for evaluating execution quality. Using these criteria, we develop a decoupled refinement framework that improves execution quality without modifying or retraining the base VLA policy. We formalize Elegant Execution as the satisfaction of Implicit Task Constraints (ITCs) and train an Elegance Critic via offline Calibrated Q-Learning to estimate the expected quality of candidate actions. At inference time, a Just-in-Time Intervention (JITI) mechanism monitors critic confidence and intervenes only at decision-critical moments, providing selective, on-demand refinement. Experiments on LIBERO-Elegant and real-world manipulation tasks show that the learned Elegance Critic substantially improves execution quality, even on unseen tasks. The proposed model enables robotic control that values not only whether tasks succeed, but also how they are performed.

Figures (10)

• Figure 1: Overview of our motivation, benchmark, and method. (Top left) Due to the mixed-quality nature of human demonstrations, VLA models exhibit correspondingly mixed-quality execution at inference time. (Top right) The proposed LIBERO-Elegant Benchmark introduces explicit Success and Elegance Criteria, enabling an Elegance-Enriched Dataset and the training of an Elegance Critic for assessing execution quality. (Bottom) At inference time, our Just-in-Time Intervention (JITI) mechanism monitors the critic’s confidence and selectively refines the policy at critical moments, providing non-invasive, value-guided improvements in execution elegance.
• Figure 2: Illustration of the proposed Just-in-Time Intervention (JITI) process. At each timestep, the Elegance Critic monitors its predicted value. When confidence remains stable (non-critical moment), the default action from the base policy is executed directly. When significant value fluctuation is detected, JITI triggers multi-sample evaluation and selects the action with the highest predicted elegance.
• Figure 3: Training pipeline of the Elegance Critic. Samples from the Elegance-Enriched Dataset are first processed by the frozen VLA backbone to obtain contextual embeddings. These embeddings, together with the corresponding actions and graded rewards, are then fed into the Calibrated Q-Learning (Cal-QL) module, which refines them to update the critic.
• Figure 4: Ablation on the Just-in-Time Intervention (JITI) mechanism. (a) Elegant Success Rate (ESR, %) across eight LIBERO-Elegant tasks, comparing the base policy, Full-Guidance, and our JITI-guided variant. (b) Average Intervention Count per episode for Full-Guidance and JITI. JITI achieves higher ESR with significantly fewer interventions, demonstrating its efficiency and selectivity.
• Figure 5: Real-world validation on our six-task suite. (a) Task setups for the six tasks. (b) Elegant Success Rate (ESR %) of the Base Policy (SmolVLA) and Ours (JITI).