RoboCAP: Robotic Classification and Precision Pouring of Diverse Liquids and Granular Media with Capacitive Sensing

TL;DR

RoboCAP leverages end-effector capacitive sensing to identify substances inside opaque containers and to achieve precise pouring with a model-based controller. The system integrates two 5-electrode sensing panels on a parallel-jaw gripper and introduces the Poured Weight Predictor (PWP) and Overpoured Weight Estimation (OWE) to estimate poured mass and prevent overpouring, respectively, achieving a mean pouring error of $3.2$ g across five substrates. On a dataset of $81$ substance-container classes, RoboCAP attains high classification accuracy (up to $98.6\%$ container, $95.8\%$ substance under random splits) and outperforms a behavior cloning baseline by a wide margin ($3.2$ g vs $23.9$ g mean error). These results demonstrate that capacitive sensing is a robust, occlusion-insensitive modality for perception and manipulation of liquids and granular media, with strong potential for generalization and multimodal integration in real-world robotic tasks.

Abstract

Liquids and granular media are pervasive throughout human environments, yet remain particularly challenging for robots to sense and manipulate precisely. In this work, we present a systematic approach at integrating capacitive sensing within robotic end effectors to enable robust sensing and precise manipulation of liquids and granular media. We introduce the parallel-jaw RoboCAP Gripper with embedded capacitive sensing arrays that enable a robot to directly sense the materials and dynamics of liquids inside of diverse containers, including some visually opaque. When coupled with model-based control, we demonstrate that the proposed system enables a robotic manipulator to achieve state-of-the-art precision pouring accuracy for a range of substances with varying dynamics properties. Code, designs, and build details are available on the project website.

Figures (8)

• Figure 1: Our capacitive sensing RoboCAP Gripper is mounted on an xArm 7; the highlighting shows two sensing arrays and their electric fields.
• Figure 2: Two 3D-printed panels, each consisting of five copper electrodes and a Teensy LC, are shown mounted on the UFactory xArm gripper. The experimental setup is shown with a weighing scale to measure ground truth for robotic pouring experiments.
• Figure 3: Capacitance data over time is shown during a grasping cycle of PP plastic and glass containers, containing water and oil. As the RoboCAP Gripper closes around the container, the sensor values increase. Each line on the plot is min-max normalized and corresponds to one of ten electrodes on the sensing arrays.
• Figure 4: RoboCAP Controller pipeline. The Poured Weight Predictor (PWP) model takes 0.1 s windows of normalized capacitance signals and the substance class as input; the outputs are the estimated change in weight per 0.1 s, $\Delta\hat{w}$, and two offset terms, $O_S$, and $O_E$. The Overpoured Weight Estimation (OWE) function takes the target weight $w_{target}$ as input and calculates the stop weight $w_{stop}$. Our controller accumulates the predicted $\Delta\hat{w}$ during the pouring process and compares the accumulated weight $\hat{w}$ with $w_{stop}$. Once the accumulated weight $\hat{w}$ exceeds $w_{stop}$, our controller starts to rotate back.
• Figure 5: The upper plots show the normalized capacitance over time for water and rice during the pouring process with a target weight of 75g. The red dashed line indicates the moment when the arm starts to pour the substance from the container. The blue dashed line indicates the moment when pouring stops and the RoboCAP Gripper starts to rotate backward. The green dashed line indicates the moment when the substance stops being poured out from the container. The three images below the graph correspond to the experimental pouring images of rice at these three stages.