DNA Origami Nanostructures Observed in Transmission Electron Microscopy Images can be Characterized through Convolutional Neural Networks

TL;DR

This paper addresses the challenge of determining the ligation number of DNA origami nanostructures bound to quantum dots in TEM images. It introduces a transfer learning workflow that pre-trains CNNs on a large set of CG MD-generated images and then fine-tunes them on a small TEM dataset to classify ligation numbers. Among nine CNN architectures, pre-trained ResNet50 and VGG16 perform best on MD test images, while fine-tuned VGG16 yields the highest agreement on TEM images, demonstrating a practical path to rapid, bias-reduced TEM analysis. The approach enables scalable, automated characterization of complex nanostructures in large TEM images and provides publicly available code and data for reuse.

Abstract

Artificial intelligence (AI) models remain an emerging strategy to accelerate materials design and development. We demonstrate that convolutional neural network (CNN) models can characterize DNA origami nanostructures employed in programmable self-assembling, which is important in many applications such as in biomedicine. Specifically, we benchmark the performance of 9 CNN models -- viz. AlexNet, GoogLeNet, VGG16, VGG19, ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152 -- to characterize the ligation number of DNA origami nanostructures in transmission electron microscopy (TEM) images. We first pre-train CNN models using a large image dataset of 720 images from our coarse-grained (CG) molecular dynamics (MD) simulations. Then, we fine-tune the pre-trained CNN models, using a small experimental TEM dataset with 146 TEM images. All CNN models were found to have similar computational time requirements, while their model sizes and performances are different. We use 20 test MD images to demonstrate that among all of the pre-trained CNN models ResNet50 and VGG16 have the highest and second highest accuracies. Among the fine-tuned models, VGG16 was found to have the highest agreement on the test TEM images. Thus, we conclude that fine-tuned VGG16 models can quickly characterize the ligation number of nanostructures in large TEM images.

Figures (10)

• Figure 1: Scheme for predicting unknown nanostructures from a TEM image: at left, the AI models are pre-trained using a large number of CG MD simulation images; at right, the models are fine-tuned by the TL method using a small amount of experimental TEM images; and at center, unknown nanostructures in the TEM images are predicted using the fine-tuned models. In the CG MD model on the left, hern24g the DNA origamis are in green, biotin binding sites are in blue, the QD is in gray, and SAv is in red. The biotin bind sites are not visible because they are embedded into SAv.
• Figure 2: Comparison of the 4 benchmarking CNN architectures for image classification: AlexNet, VGG, GoogLeNet and ResNet. VGG19 has 3 more CL than VGG16. ResNet18, ResNet50, ResNet101 and ResNet152 vary the number of CL with the same architecture as ResNet34. The CL4 to CL21 in GoogLeNet use 'kernel $1\times1$, stride 1 and padding 0', 'kernel $3\times3$, stride 1 and padding 1', and 'kernel $5\times5$, stride 1 and padding 2', 3 different CL. The CL2 to CL 33 in ResNet34 all use 'kernel $3\times3$, stride 1 and padding 1'. The larger ResNets --- ResNet50, ResNet101 and ResNet152 --- use both 'kernel $1\times1$, stride 1 and padding 0' and 'kernel $3\times3$, stride 1 and padding 1' in CL.
• Figure 3: Comparison of total cost, model size and parameter size across the 9 CNN--- viz. AlexNet, VGG16, VGG19, GoogLeNet, ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152---in the bottom, middle and top panels, respectively. The reported average computing time cost corresponds to the completion of 90 epochs of training and validation runs on the MD dataset using 1 NVIDIA A100 40 GB GPU.
• Figure 4: Validation accuracies among different ML and AI models--- viz.DT,song15RF,breiman01 AlexNet, VGG16, VGG19, GoogLeNet, ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152---after 90 training epochs.
• Figure 5: The simulated test dataset consists of the twenty images, 1-20, obtained from projections of CG MD 3D models. The ground truth for each image--- viz. the ligation number---is listed in the table on the right.