Sashimi-Bot: Autonomous Tri-manual Advanced Manipulation and Cutting of Deformable Objects

TL;DR

Sashimi-Bot tackles the challenge of autonomous manipulation of deformable, volumetric objects by introducing a tri-arm system that combines non-prehensile shape ser voing, conventional cutting, and delicate picking. The approach integrates a DRL-based shape manipulation module, a cutting planner with tactile feedback from GelSight sensors, and a vision-based picker, enabling end-to-end sashimi preparation with zero-shot sim-to-real transfer. Key contributions include a boundary-transformer-based DRL policy for shape manipulation, tactile-guided cutting with real-time trajectory adjustment, and a robust 4-DoF visual servoing picker. The results demonstrate autonomous shaping, cutting, and serving of sashimi across variable loins, highlighting the system’s potential to generalize to other deformable-object tasks and real-world manufacturing or food-processing applications.

Abstract

Advanced robotic manipulation of deformable, volumetric objects remains one of the greatest challenges due to their pliancy, frailness, variability, and uncertainties during interaction. Motivated by these challenges, this article introduces Sashimi-Bot, an autonomous multi-robotic system for advanced manipulation and cutting, specifically the preparation of sashimi. The objects that we manipulate, salmon loins, are natural in origin and vary in size and shape, they are limp and deformable with poorly characterized elastoplastic parameters, while also being slippery and hard to hold. The three robots straighten the loin; grasp and hold the knife; cut with the knife in a slicing motion while cooperatively stabilizing the loin during cutting; and pick up the thin slices from the cutting board or knife blade. Our system combines deep reinforcement learning with in-hand tool shape manipulation, in-hand tool cutting, and feedback of visual and tactile information to achieve robustness to the variabilities inherent in this task. This work represents a milestone in robotic manipulation of deformable, volumetric objects that may inspire and enable a wide range of other real-world applications.

Figures (10)

• Figure 1: Sashimi-Bot is a trimanual robotic framework capable of autonomously preparing sashimi. Left: the system in action. Right: starting with a salmon loin in an arbitrary initial configuration (I), the high-level components of the system involve shape manipulation (II), cutting (III), and picking and placing (IV).
• Figure 2: High-level components of the Sashimi-Bot pipeline. a) A pointcloud of the workspace is provided by a static, top-down camera. For shape manipulation, we present b) a non-prehensile method based on Deep Reinforcement Learning (DRL). c) Cutting trajectories are generated by a motion planning algorithm with d) tactile feedback from GelSight sensors. e) The pick-and-place module uses feedback from a separate, wrist-mounted camera and Visual Servoing (VS) to grasp sashimi pieces with a pair of chopsticks.
• Figure 3: DRL-based shape manipulation results. a) Number of manipulation actions (steps) needed to reach convergence for each initial shape, L, C, Z, across 10 randomized trials. b) Sample straightening of a Z shaped object.
• Figure 4: Cutting results across different configurations. We vary whether to cut with the knife perpendicular to the cutting board or tilted ($\sim20$ degrees), and whether to cut with a single backward stroke or dual forward-backward strokes
• Figure 5: Tactile feedback results. a) Sample GelSight sensor images from non-contact and contact with the cutting board, and the difference between the two. b) Predicted probability of board contact in four different scenarios. Dashed vertical line indicates first point in time where $\text{p(board)} \geq 0.5$. The first row shows model response for regular cuts through a salmon loin that end up in contact with the board, the second row shows cutting paths that only go through the salmon without hitting the board, the third and forth rows repeat these experiments without a salmon loin (cutting air). Note that contact is only detected when the knife actually hits the cutting board. c) Example of closed-loop trajectory adjustments based on tactile feedback. We raise the board height to provoke early contact with the knife. When contact is detected, the remaining trajectory is adjusted so that no point on the knife will ever go below the vertical position of the contact point. See the end of Section \ref{['sec:gelsight-feedback-appendix']} for more details.