MORPHeus: a Multimodal One-armed Robot-assisted Peeling System with Human Users In-the-loop

TL;DR

MORPHeus tackles the challenging problem of robotic peeling with a single arm by integrating a multimodal perception system, human-in-the-loop long-horizon planning, and a compliant Cartesian impedance controller. The pipeline uses visual, force, and vibration data to determine peeled regions, an LLM-driven planning interface with PDDL to respect user peeling preferences, and a safety-conscious impedance control to adapt to varying food properties. Key contributions include (1) a one-armed peeling system supported by an open-hardware multimodal peeler, (2) a perception model that fuses multiple sensing modalities to classify peeled state, (3) a GPT-4 + PDDL planning framework with anomaly detection for robust long-horizon task execution, and (4) comprehensive evaluation across 12 diverse foods demonstrating practical feasibility. The work advances home-care robotics by enabling flexible, preference-aware peeling with a single robotic arm, offering a viable path toward accessible, autonomous meal-preparation assistance for individuals with mobility limitations.

Abstract

Meal preparation is an important instrumental activity of daily living~(IADL). While existing research has explored robotic assistance in meal preparation tasks such as cutting and cooking, the crucial task of peeling has received less attention. Robot-assisted peeling, conventionally a bimanual task, is challenging to deploy in the homes of care recipients using two wheelchair-mounted robot arms due to ergonomic and transferring challenges. This paper introduces a robot-assisted peeling system utilizing a single robotic arm and an assistive cutting board, inspired by the way individuals with one functional hand prepare meals. Our system incorporates a multimodal active perception module to determine whether an area on the food is peeled, a human-in-the-loop long-horizon planner to perform task planning while catering to a user's preference for peeling coverage, and a compliant controller to peel the food items. We demonstrate the system on 12 food items representing the extremes of different shapes, sizes, skin thickness, surface textures, skin vs flesh colors, and deformability.

Figures (7)

• Figure 1: Concept of robot-assisted peeling with an adaptive cutting board: (1, 2) The robot uses fixtures on the board to secure the food and interacts with the food, resulting in long-horizon manipulation. (3) The robot uses a peeler instrumented with multimodal sensors to identify whether an area is peeled.
• Figure 2: Overview of the building blocks of MORPHeus: Multimodal perception (Sec. \ref{['Sec:multimodal']}) module uses visual, vibration, and force inputs to determine whether the food is peeled, Human-in-the-loop Long-horizon Planning (Sec. \ref{['Planning:Integrating']}) enables the robot to perform task planning while catering to a user's preference and perform anomaly detection using user's feedback, and Compliant Control (Sec. \ref{['Sec:control']}) to execute peeling actions for various food items.
• Figure 3: Model structure for multimodal perception: The network uses image, vibration, and force as input, and gives a binary output for peeled/unpeeled classification ($O$).
• Figure 4: (a) Actions the robot can take. For bidirectional actions including SlideIn/Out, Lock/Unlock, MoveToJaws/MoveFromJaws, we only visualize one direction here. (b) Various types of cutting boards. (c) The experiment setup includes a Franka Emika Panda 7 DoF Robot Arm, an assistive cutting board with a moving jaw, a 3D printed peeler holder, and a peeler with multimodal perception for vision, force, and vibration modalities.
• Figure 5: (a) Users' preference: By expressing their preference for the food to be 'Half-peeled' or 'Fully-peeled', the users can get their food peeled according to their preferred peeling goal. (b) 6 axes corresponding to the characteristics of different food items and 12 food types selected to represent the extremes. (c) Food items peeled by MORPHeus.