Learning Adaptive Force Control for Contact-Rich Sample Scraping with Heterogeneous Materials

TL;DR

An adaptive control framework is proposed, relying on a low-level Cartesian impedance controller for stable and compliant physical interaction and a high-level reinforcement learning agent that learns to dynamically adjust interaction forces at the end-effector.

Abstract

The increasing demand for accelerated scientific discovery, driven by global challenges, highlights the need for advanced AI-driven robotics. Deploying robotic chemists in human-centric labs is key for the next horizon of autonomous discovery, as complex tasks still demand the dexterity of human scientists. Robotic manipulation in this context is uniquely challenged by handling diverse chemicals (granular, powdery, or viscous liquids), under varying lab conditions. For example, humans use spatulas for scraping materials from vial walls. Automating this process is challenging because it goes beyond simple robotic insertion tasks and traditional lab automation, requiring the execution of fine-granular movements within a constrained environment (the sample vial). Our work proposes an adaptive control framework to address this, relying on a low-level Cartesian impedance controller for stable and compliant physical interaction and a high-level reinforcement learning agent that learns to dynamically adjust interaction forces at the end-effector. The agent is guided by perception feedback, which provides the material's location. We first created a task-representative simulation environment with a Franka Research 3 robot, a scraping tool, and a sample vial containing heterogeneous materials. To facilitate the learning of an adaptive policy and model diverse characteristics, the sample is modelled as a collection of spheres, where each sphere is assigned a unique dislodgement force threshold, which is procedurally generated using Perlin noise. We train an agent to autonomously learn and adapt the optimal contact wrench for a sample scraping task in simulation and then successfully transfer this policy to a real robotic setup. Our method was evaluated across five different material setups, outperforming a fixed-wrench baseline by an average of 10.9%.

Figures (6)

• Figure 1: The proposed control architecture for autonomous sample scraping. A high-level RL policy receives a visual state ($s_{\text{p}}=[v_1,v_2,v_3]^T$), with $v_i=[c_{ix},c_{iy},c_{iz},p_i]^T$ denoting each cluster's centroid and residue percentage) and the robot's Cartesian state with the external wrench ($x, \dot{x}, F_{\text{ext}}$). The policy outputs a hybrid action command ($\boldsymbol{a}_t = [f_x^c, \tau_y^c, z^D]^T$) at 10 Hz, defining a desired force, torque, and position in the z-axis. This command is used as a goal to the Cartesian impedance controller running at 500 Hz, which generates the compliant joint torques ($\tau_c$).
• Figure 2: The sequential phases of the autonomous in-vial sample scraping task: (left) tool insertion with lateral force $F_x$ and torque $\tau_y$ applied against the vial wall, with RGB-D camera capturing visual state $s_p$; (centre) RL policy adapts $F_x$, $\tau_y$, and vertical position $z^{D}$ based on real-time perception feedback $s_p$, controlling both contact wrench and scraping direction; (right) final upward sweep with adjusted $F_x$ to clear residual material, with dislodged particles settled at the bottom of the vial.
• Figure 3: The proposed perception pipeline. From RGB-D input, YOLO Redmon2016 localises the vial, segmented via GrabCut Chen2008. Depth filtering isolates front-facing material and colour-based filtering removes the spatula. K-means clustering over a region of interest (ROI) then yields the material clusters, each represented by a centroid ($c_x, c_y, c_z$) and residue percentage ($p$).
• Figure 4: A dynamic depth threshold method to detect the presence of material closest to the camera. The threshold is calculated by interpolating between the closest and furthest depth measurements based on a user-defined ratio (e.g., a ratio of 0.5 would correspond to the front half of the vial.)
• Figure 5: An overview of the simulation environment and real robot setup, including the robotic manipulator (Franka Research 3), the scraping tool and the vial.