A Self-Supervised Robotic System for Autonomous Contact-Based Spatial Mapping of Semiconductor Properties

TL;DR

The proposed approach for building self-supervised autonomy into contact-based robotic systems that teach the robot to follow domain expert measurement principles at high throughputs enables high-precision and reliable automation of contact-based characterization techniques at high throughputs, thereby allowing measurement of previously inaccessible yet important semiconductor properties for self-driving laboratories.

Abstract

Integrating robotically driven contact-based material characterization techniques into self-driving laboratories can enhance measurement quality, reliability, and throughput. While deep learning models support robust autonomy, current methods lack reliable pixel-precision positioning and require extensive labeled data. To overcome these challenges, we propose an approach for building self-supervised autonomy into contact-based robotic systems that teach the robot to follow domain expert measurement principles at high-throughputs. Firstly, we design a vision-based, self-supervised convolutional neural network (CNN) architecture that uses differentiable image priors to optimize domain-specific objectives, refining the pixel precision of predicted robot contact poses by 20.0% relative to existing approaches. Secondly, we design a reliable graph-based planner for generating distance-minimizing paths to accelerate the robot measurement throughput and decrease planning variance by 6x. We demonstrate the performance of this approach by autonomously driving a 4-degree-of-freedom robotic probe for 24 hours to characterize semiconductor photoconductivity at 3,025 uniquely predicted poses across a gradient of drop-casted perovskite film compositions, achieving throughputs over 125 measurements per hour. Spatially mapping photoconductivity onto each drop-casted film reveals compositional trends and regions of inhomogeneity, valuable for identifying manufacturing process defects. With this self-supervised CNN-driven robotic system, we enable high-precision and reliable automation of contact-based characterization techniques at high throughputs, thereby allowing the measurement of previously inaccessible yet important semiconductor properties for self-driving laboratories.

Figures (13)

• Figure 1: Using domain information to guide robot autonomy. a, A materials science domain expert chooses the positions and angles of three distinct contact points (red) of a probe to measure a semiconductor film, ensuring maximum coverage while avoiding overlap. Using vision-guided deep learning, this domain information is embedded into the model's objective-based loss functions. b, A 4-degree-of-freedom robotic probe trained using this domain information-embedded deep learning model emulates the measurement procedure of a domain expert autonomously.
• Figure 2: Self-supervised robotic approach for autonomous contact-based characterization of semiconductors. a, Synthesized and annealed drop-casted semiconductor films are fed into the autonomous process, which begins with computer vision segmentation of the semiconductor films, then the prediction of optimal robot poses using a spatially differentiable CNN (SDCNN), distance-minimizing path planning, and finally robotic control with subsequent measurement. b, Photoconductivity and surface profilometry use-cases of the robotic contact-based characterization process. c, Average positional and rotational accuracy across $3,500$ robot pose predictions with two standard deviation error for our SDCNN method, compared against seven other models.
• Figure 3: Implementing CNN spatial differentiability for self-supervised robot pose optimization. a, Image segment pixels ($I_X, I_Y$) are passed through a Gaussian filter to maintain the differentiability of the edges. Predicted pose pixels, $(\mathrm{Pose}_X, \mathrm{Pose}_Y)$, from the SDCNN are superimposed onto the differentiable segment pixels, ($I'_X, I'_Y$), using another Gaussian filter and sigmoid function, $S(\cdot)$, to perform direct computation and back-propagation in image space. b, SDCNN architecture and differential predicted poses, $(\mathrm{Pose}'_X, \mathrm{Pose}'_Y)$, composed onto the image segment pixels, ($I'_X, I'_Y$). The SDCNN is trained to predict poses onto the measurable regions of the films, sufficiently distanced from its edges.
• Figure 4: Pose prediction performance across CNN models on drop-casted semiconductor films with varying geometries. a, Loss minimization procedure in image space using our spatially differentiable CNN (SDCNN). b, Predicted poses for eight different CNN models on a subset of nine experimentally synthesized semiconductor films for $k=3$ poses per film. Predicted valid contact poses are shown in white, and invalid poses are shown in red. The photocurrent curve at each valid pose is measured experimentally using the robotic system. c, Left: stochastic process predictions. Middle: prediction success rate on the full set of 35 experimentally synthesized semiconductors across 100 unique trials. Right: CNN inference time on the pose prediction task for the full set of films across 100 unique trials, run on an NVIDIA Tesla V100 GPU.
• Figure 5: Distance-minimizing path planning performance across graph-based algorithms for autonomous contact-based robotics. a--e, Demonstration of the planning mechanics for five path-finding algorithms to solve an Open Loop Traveling Salesman Problem (OTSP). A path plan generated by each of these methods is illustrated by the color map, indicating the path direction for a graph of nodes, each node representing a robot contact pose predicted by the spatially differentiable CNN (SDCNN) on an array of drop-casted semiconductor films. f, The robot following a path generated by our proposed noisy Dijkstra path planner. g, Left: total path lengths of generated routes across $115$ independent trials, each computed on a unique graph of $k=3$ SDCNN-predicted robot contact poses for an array of 35 drop-casted semiconductor films. Right: compute time to generate a path plan across $115$ independent trials, run on an NVIDIA GeForce RTX 4090 GPU.