Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Adaptable, Safe, and Portable Robot-Assisted Feeding System

Ethan Kroll Gordon, Rajat Kumar Jenamani, Amal Nanavati, Ziang Liu, Haya Bolotski, Raida Karim, Daniel Stabile, Atharva Kashyap, Bernie Hao Zhu, Xilai Dai, Tyler Schrenk, Jonathan Ko, Taylor Kessler Faulkner, Tapomayukh Bhattacharjee, Siddhartha Srinivasa

TL;DR

The paper tackles self-feeding for people with mobility impairments by presenting a safe, portable, and user-controlled robot-assisted feeding system that mounts on a wheelchair and is operable via a web app. It integrates Kinova Gen2/Gen3 arms with onboard compute, real-time sensing, and ROS2-based software (MoveIt2, behavior trees) to manage bite acquisition and bite transfer. A $26$-dimensional bite acquisition schema with $11$ discrete online actions is learned via a contextual bandit, supported by real-time multi-view mouth perception and interaction-aware control for safe in-mouth hand-offs, including an offline/online safety-focused design. The system is validated through extensive end-user studies across multiple conditions and impairments, demonstrating potential to reduce caregiver burden and promote independence for users with MS, SCI, SMA, and Arthrogryposis, with potential for broad real-world deployment.

Abstract

We demonstrate a robot-assisted feeding system that enables people with mobility impairments to feed themselves. Our system design embodies Safety, Portability, and User Control, with comprehensive full-stack safety checks, the ability to be mounted on and powered by any powered wheelchair, and a custom web-app allowing care-recipients to leverage their own assistive devices for robot control. For bite acquisition, we leverage multi-modal online learning to tractably adapt to unseen food types. For bite transfer, we leverage real-time mouth perception and interaction-aware control. Co-designed with community researchers, our system has been validated through multiple end-user studies.

An Adaptable, Safe, and Portable Robot-Assisted Feeding System

TL;DR

The paper tackles self-feeding for people with mobility impairments by presenting a safe, portable, and user-controlled robot-assisted feeding system that mounts on a wheelchair and is operable via a web app. It integrates Kinova Gen2/Gen3 arms with onboard compute, real-time sensing, and ROS2-based software (MoveIt2, behavior trees) to manage bite acquisition and bite transfer. A -dimensional bite acquisition schema with discrete online actions is learned via a contextual bandit, supported by real-time multi-view mouth perception and interaction-aware control for safe in-mouth hand-offs, including an offline/online safety-focused design. The system is validated through extensive end-user studies across multiple conditions and impairments, demonstrating potential to reduce caregiver burden and promote independence for users with MS, SCI, SMA, and Arthrogryposis, with potential for broad real-world deployment.

Abstract

We demonstrate a robot-assisted feeding system that enables people with mobility impairments to feed themselves. Our system design embodies Safety, Portability, and User Control, with comprehensive full-stack safety checks, the ability to be mounted on and powered by any powered wheelchair, and a custom web-app allowing care-recipients to leverage their own assistive devices for robot control. For bite acquisition, we leverage multi-modal online learning to tractably adapt to unseen food types. For bite transfer, we leverage real-time mouth perception and interaction-aware control. Co-designed with community researchers, our system has been validated through multiple end-user studies.
Paper Structure (10 sections, 3 figures)

This paper contains 10 sections, 3 figures.

Table of Contents

  1. Motivation and System Overview
  2. Hardware
  3. Software
  4. Design and Technical Details
  5. Design Principle: Safety
  6. Design Principle: Portability
  7. Design Principle: User Control
  8. Bite Acquisition
  9. Bite Transfer
  10. Funding Sources

Figures (3)

  • Figure 1: (Left) Diagram of the system logic. The user drives the app, which calls an API on the robot computer. (Right) Hardware system for both the Gen2 and Gen3 base. No external wires are needed.
  • Figure 2: Bite acquisition: an action space derived from human data for online learning within a contextual bandit framework gordon2023schema.
  • Figure 3: Bite transfer: multi-view mouth perception and physical interaction-aware control for an in-mouth hand-off jenamani2024bitetransfer.