CRAFT: Adapting VLA Models to Contact-rich Manipulation via Force-aware Curriculum Fine-tuning

TL;DR

CRAFT addresses the gap in VLA models for contact-rich manipulation by balancing force signals with high-entropy vision and language inputs through a force-aware curriculum fine-tuning framework. It introduces a variational information bottleneck that compresses vision–language representations to prioritize joint torque proprioception during early training, then gradually reintroduces visual and linguistic cues while preserving force-aware representations. The approach is model-agnostic and demonstrated on pi_0 and RDT using a homologous leader–follower teleoperation data collection system to obtain synchronized vision, language, and force data. Real-world experiments show improved task success, stronger generalization to unseen objects and task variations, and broad applicability across VLA architectures, highlighting the practical impact of force-aware, information-theoretic modulation in robotics.

Abstract

Vision-Language-Action (VLA) models have shown a strong capability in enabling robots to execute general instructions, yet they struggle with contact-rich manipulation tasks, where success requires precise alignment, stable contact maintenance, and effective handling of deformable objects. A fundamental challenge arises from the imbalance between high-entropy vision and language inputs and low-entropy but critical force signals, which often leads to over-reliance on perception and unstable control. To address this, we introduce CRAFT, a force-aware curriculum fine-tuning framework that integrates a variational information bottleneck module to regulate vision and language embeddings during early training. This curriculum strategy encourages the model to prioritize force signals initially, before progressively restoring access to the full multimodal information. To enable force-aware learning, we further design a homologous leader-follower teleoperation system that collects synchronized vision, language, and force data across diverse contact-rich tasks. Real-world experiments demonstrate that CRAFT consistently improves task success, generalizes to unseen objects and novel task variations, and adapts effectively across diverse VLA architectures, enabling robust and generalizable contact-rich manipulation.

Figures (6)

• Figure 1: Homologous leader–follower teleoperation system. The operator controls the leader arm while the follower arm mirrors the motion. Real-time force feedback enables natural interaction sensing, and two synchronized cameras capture first-person and third-person visual observations.
• Figure 2: Overview of CRAFT. CRAFT mitigates the imbalance between high-entropy perceptual signals (vision and language) and low-entropy force signals in contact-rich manipulation. The VIB module initially compresses visual and language features to emphasize force information, and a curriculum weight schedule gradually relaxes this constraint, allowing the policy to integrate rich multimodal signals while retaining force-aware representations. CRAFT can be applied to diverse VLA models.
• Figure 3: Representative contact-rich tasks. (1) USB Insertion, (2) Flip Carton, (3) Wipe Whiteboard, (4) Rolling Plasticine, and (5) Shaft-to-Hole Insertion.
• Figure 4: Task success rates of baselines and CRAFT-enhanced VLA models. Across five representative manipulation tasks, CRAFT consistently improves success rates for both $\pi_0$-base and RDT. The gains are particularly significant for contact-rich tasks, demonstrating the effectiveness of our force-aware curriculum fine-tuning strategy.
• Figure 5: CRAFT enables force-aware manipulation. Comparison of $\pi_0$-base (a) and CRAFT-enhanced (b) on USB insertion and flip carton. CRAFT uses force signals to adjust motions, enabling successful execution.