CulinaryCut-VLAP: A Vision-Language-Action-Physics Framework for Food Cutting via a Force-Aware Material Point Method

TL;DR

CulinaryCut-VLAP tackles the challenge of grounding vision-language-action policies in physically realistic deformable cutting by coupling a large-scale VLA-style dataset with an MLS-MPM-based cutting simulator. The framework combines a Vision-Language-Action Model, a Manipulation Safety Module, and a Cutting Style Transfer Module to produce force-aware, style-consistent cutting actions under topology-changing interactions. The benchmark comprises 325k simulation trajectories across 7 foods, 5 cut styles, and 13 cut states, with multi-view visuals and force/pose labels to enable sim-to-real transfer and rigorous generalization testing. Ablation studies demonstrate the critical roles of topology-aware data, safety constraints, and continuous-ratio grounding for stable and safe cutting policies in cluttered and diverse scenarios. Overall, the work provides a physically grounded, scalable foundation for VLA-based deformable-object manipulation with practical implications for robotics and culinary automation.

Abstract

Food cutting is a highly practical yet underexplored application at the intersection of vision and robotic manipulation. The task remains challenging because interactions between the knife and deformable materials are highly nonlinear and often entail large deformations, frequent contact, and topological change, which in turn hinder stable and safe large-scale data collection. To address these challenges, we propose a unified framework that couples a vision-language-action (VLA) dataset with a physically realistic cutting simulator built on the material point method (MPM). Our simulator adopts MLS-MPM as its computational core, reducing numerical dissipation and energy drift while preserving rotational and shear responses even under topology-changing cuts. During cutting, forces and stress distributions are estimated from impulse exchanges between particles and the grid, enabling stable tracking of transient contact forces and energy transfer. We also provide a benchmark dataset that integrates diverse cutting trajectories, multi-view visual observations, and fine-grained language instructions, together with force--torque and tool--pose labels to provide physically consistent training signals. These components realize a learning--evaluation loop that respects the core physics of cutting and establishes a safe, reproducible, and scalable foundation for advancing VLA models in deformable object manipulation.

Figures (15)

• Figure 1: Concept of the Vision–Language–Action-Physics (VLAP) framework. The framework interacts between the robot and physics simulation to construct a physically grounded VLA dataset and model for food cutting.
• Figure 2: Overview of CulinaryCut data generation pipeline. (a) Teleoperation generates initial cutting demonstrations for each cut style. (b) Physics simulation updates material deformation and cutting geometry. (c) Language instructions are augmented using LLMs with continuous cut state variations. (d, e) Trajectories are augmented via motion planning with object and cutting randomization.
• Figure 3: Overall pipeline of the Vision-Language-Action-Physics (VLAP) model during inference on the CulinaryCut dataset.
• Figure 4: Object Variation Results. The bar chart shows the inference performance of each model when trained on individual object–instruction pairs.
• Figure 5: Multi-Object Results: Comparison of RDT, Octo, and OpenVLA in single- and multi-object scenes. The presence of additional objects significantly impairs target identification and reduces cutting success rates.