GRATEV2.0: Computational Tools for Real-time Analysis of High-throughput High-resolution TEM (HRTEM) Images of Conjugated Polymers

TL;DR

GRATEv2 provides an open-source, real-time HRTEM analysis framework for conjugated polymers that combines fast image-processing with Bayesian parameter optimization and a Wasserstein-distance stopping criterion to optimize data collection. The approach enables automated tuning of 13 material-specific processing parameters, robust segmentation, and scalable HPC-enabled throughput, demonstrated on the PCDTBT dataset with 4350 crystalline domains detected. By extracting features such as $d$-spacing, orientation, and crystal shape metrics, the method advances high-throughput nanoscale characterization in organic electronics. The combination of automated tuning, quantitative stopping criteria, and open-source availability supports reproducibility and broad adoption for materials discovery and optimization.

Abstract

Automated analysis of high-resolution transmission electron microscopy (HRTEM) images is increasingly essential for advancing research in organic electronics, where precise characterization of nanoscale crystal structures is crucial for optimizing material properties. This paper introduces an open-source computational framework called GRATEV2.0 (GRaph-based Analysis of TEM), designed for real-time analysis of HRTEM data, with a focus on characterizing complex microstructures in conjugated polymers, illustrated using Poly[N-9'-heptadecanyl-2,7-carbazole-alt-5,5-(4',7'-di-2-thienyl-2',1',3'-benzothiadiazole)] (PCDTBT), a key material in organic photovoltaics. GRATEV2.0 employs fast, automated image processing algorithms, enabling rapid extraction of structural features like d-spacing, orientation, and crystal shape metrics. Gaussian process optimization rapidly identifies the user-defined parameters in the approach, reducing the need for manual parameter tuning and thus enhancing reproducibility and usability. Additionally, GRATEV2.0 is compatible with high-performance computing (HPC) environments, allowing for efficient, large-scale data processing at near real-time speeds. A unique feature of GRATEV2.0 is a Wasserstein distance-based stopping criterion, which optimizes data collection by determining when further sampling no longer adds statistically significant information. This capability optimizes the amount of time the TEM facility is used while ensuring data adequacy for in-depth analysis. Open-source and tested on a substantial PCDTBT dataset, this tool offers a powerful, robust, and accessible solution for high-throughput material characterization in organic electronics.

Figures (14)

• Figure 1: Schematic overview of the GRATEv2 computational framework. The framework processes raw HRTEM images of PCDTBT, applies preprocessing, and performs automated image processing with parameters optimized via Bayesian optimization. A data sufficiency criterion based on the Wasserstein distance assesses whether additional TEM data is needed. The output comprises extracted structural features such as d-spacing, orientation, and crystal shape metrics. (a) The overall computational framework of GRATEv2, and (b) the detailed Bayesian Optimization (BO) component used for parameter tuning.
• Figure 2: Comparison between (a) the original HRTEM image and (b) the manually annotated crystals used for training the Bayesian model. 13 manually annotated images are used for effective training in the work. Manual annotation is time-consuming and subjective.
• Figure 3: Comparison of Ground Truth, Manually selected parameter detection, and Bayesian-Optimized parameter detection across three different images. Each column represents a distinct detection method, illustrating how Bayesian optimization enhances segmentation accuracy by more closely matching the ground truth annotations compared to manual parameter tuning. The Bayesian optimization process was conducted over 200 iterations, achieving a minimum loss value of -0.7319 at the 151st evaluation.
• Figure 4: Convergence of the Bayesian optimization process over 200 Iterations, illustrating the reduction in the loss function (negative IoU) as optimization progresses. The y-axis represents the minimum loss value achieved up to that evaluation. The minimum loss value of -0.7319 was attained at the 151st evaluation.
• Figure 5: (a) and (b) are 1.tif and 2.tif images respectively corresponding to \ref{['table:HRTEMDetectionResults']} and \ref{['table:HRTEMCrystalCorrelation']}. (a) and (b) shows the original image (left) and the segmentation output (right) from our algorithm for HRTEM of PCDTBT. The detected crystals have a d-spacing of 1.9nm. The image on the left is the input to the algorithm, and on the right is the output of the algorithm. Each detected crystal in the output shows (1) the convex hull boundary around the crystal, (2) the shaded region representing a more exact crystal region, and (3) a straight line at the centroid of the convex hull, which shows the orientation of the crystal patterns.