Mind the Gap: Bridging the Divide Between AI Aspirations and the Reality of Autonomous Characterization

TL;DR

The paper investigates how AI can bridge aspirations with the practical realization of autonomous characterization in electron microscopy. It frames the inverse problem for microscopy as $g(x,y) = f(x,y) \otimes h(x,y)$ and argues multimodal data can constrain $f$ via diverse encoders and fusion to yield robust structural descriptors. The authors demonstrate two applications—atomically precise defect discovery in MXenes and multimodal disorder analysis in irradiated oxide interfaces—showing scalable, reproducible statistics beyond manual analysis. They also discuss practical obstacles, including encoder development, data standards, and hardware-software integration, and propose a roadmap toward co-pilots and fully autonomous microscopes enabled by open software and digital twins.

Abstract

What does materials science look like in the "Age of Artificial Intelligence?" Each materials domain-synthesis, characterization, and modeling-has a different answer to this question, motivated by unique challenges and constraints. This work focuses on the tremendous potential of autonomous characterization within electron microscopy. We present our recent advancements in developing domain-aware, multimodal models for microscopy analysis capable of describing complex atomic systems. We then address the critical gap between the theoretical promise of autonomous microscopy and its current practical limitations, showcasing recent successes while highlighting the necessary developments to achieve robust, real-world autonomy.

Figures (7)

• Figure 1: The quest for the autonomous microscope. Bridging from descriptive to prescriptive analytics is a central challenge in achieving full autonomy. Second panel adapted from Ter-Petrosyan et al.Ter-Petrosyan.10.48550/arxiv.2411.09896. Third panel adapted from Olszta et al.Olszta.10.69761/dnka1581 under CC-BY 4.0 license.
• Figure 2: Computer vision reveals atomic positions in 2D materials. Raw STEM-HAADF MXene images (left) are translated into atomic positions using a Neural Network (NN). The NN first outputs a mask (center) and then finds atomic positions (right). Through ML we are able to analyze MXene images in a more reproducible, scalable manner.
• Figure 3: Multimodal analytics informs material descriptors. (a) STEM-HAADF and STEM-EDS provide structural and compositional information on disorder in oxides. (b) Various data classification approaches show the improved discriminating power of multimodal data. (c) These classifiers inform physical descriptors for disorder, including compositional changes and crystallinity. Adapted from Ter-Petrosyan et al.Ter-Petrosyan.10.48550/arxiv.2411.09896
• Figure 4: A crystal to microscope interpreter abstracts instrument control. NanoCartographer aids in the translation from instrument to meaningful crystallographic information for imaging and diffraction data, improving the speed and reproducibility of experiments. Adapted from Olszta et al. under CC-BY 4.0 license.Olszta.10.69761/dnka1581
• Figure 5: Highlighting the gap between idealized autonomous workflows and their practical implementation. (a) Step-by-step logic for a prototype experiment in analyzing 2D materials flakes, with areas for AI improvement indicated. (b) Results of automated stage movement implemented using the AutoEM system, revealing errors in positioning and challenges in post-processing. Adapted from Fiedler et al. under CC-BY 4.0 license.Fiedler.10.1093/micmic/ozad108