Accelerating Laboratory Automation Through Robot Skill Learning For Sample Scraping

TL;DR

The paper tackles automating sample scraping in crystallisation workflows by learning a closed-loop, force-feedback policy with model-free reinforcement learning. It trains a policy in a simulated Panda Franka Emika environment with a laboratory scraper and uses Hindsight Experience Replay and curriculum learning to improve sample efficiency and generalization, then transfers the learned behavior to a real robot to autonomously scrape vial walls. Key contributions include a novel closed-space scraping benchmark, demonstration of a learning-based approach for a dexterous, contact-rich lab skill, and a curriculum-enabled transfer to real hardware across multiple vial setups. The work highlights the potential of learning-based automation to accelerate material discovery while mitigating dexterity challenges, with implications for crystallisation workflows and beyond.

Abstract

The use of laboratory robotics for autonomous experiments offers an attractive route to alleviate scientists from tedious tasks while accelerating material discovery for topical issues such as climate change and pharmaceuticals. While some experimental workflows can already benefit from automation, sample preparation is still carried out manually due to the high level of motor function and dexterity required when dealing with different tools, chemicals, and glassware. A fundamental workflow in chemical fields is crystallisation, where one application is polymorph screening, i.e., obtaining a three dimensional molecular structure from a crystal. For this process, it is of utmost importance to recover as much of the sample as possible since synthesising molecules is both costly in time and money. To this aim, chemists scrape vials to retrieve sample contents prior to imaging plate transfer. Automating this process is challenging as it goes beyond robotic insertion tasks due to a fundamental requirement of having to execute fine-granular movements within a constrained environment (sample vial). Motivated by how human chemists carry out this process of scraping powder from vials, our work proposes a model-free reinforcement learning method for learning a scraping policy, leading to a fully autonomous sample scraping procedure. We first create a scenario-specific simulation environment with a Panda Franka Emika robot using a laboratory scraper that is inserted into a simulated vial, to demonstrate how a scraping policy can be learned successfully in simulation. We then train and evaluate our method on a real robotic manipulator in laboratory settings, and show that our method can autonomously scrape powder across various setups.

Figures (7)

• Figure 1: Overview of laboratory sample scraping, where (left) a human scientist scrapes crystals that generally form on vial walls by transcending along the walls vertically and (right) our robot learning to scrape via RL.
• Figure 2: The overall block diagram of autonomous robotic scraping. Our method consists of an RL policy and a robotic controller. We map the action (cartesian pose) to joint positions using inverse kinematics to achieve the target goal. The controller uses force/torque feedback at the end effector and a joint space controller for motion planning.
• Figure 3: An illustration of the robotic scraping task. The robot inserts the scraper inside the vial. The robot's goal is to translate along the z-axis while maintaining contact with the vial wall ($F_y \geq C_{threshold}$).
• Figure 4: Evaluation results of learning a scraping policy using TQC and SAC in simulation across 5 random seeds.
• Figure 5: Evaluation results of learning a scraping policy, using TQC, with a curriculum in simulation across 5 random seeds. We can see that using curriculum learning has higher success chances with increasing task difficulty, while using no curriculum learning is unable to perform well.