3D Reconstruction of Protein Structures from Multi-view AFM Images using Neural Radiance Fields (NeRFs)

TL;DR

The paper tackles the challenge of predicting 3D structures for protein complexes from AFM data, where traditional sequence- or cryo-EM-based methods struggle. It combines UpFusion for unposed-view diffusion-based novel-view synthesis with an instance-specific NeRF optimization to yield coherent 3D reconstructions, supported by a virtual AFM imaging pipeline that generates multi-view training data from PDB structures. Zero-shot experiments on both actual and virtual AFM images demonstrate that more input views and diverse view sets enhance reconstruction quality, with PSNR, SSIM, and LPIPS metrics indicating improvements. The work highlights the potential of integrating volume rendering, diffusion-based view synthesis, and NeRF optimization to improve protein complex structure prediction, and suggests further gains from targeted fine-tuning on expansive virtual AFM datasets.

Abstract

Recent advancements in deep learning for predicting 3D protein structures have shown promise, particularly when leveraging inputs like protein sequences and Cryo-Electron microscopy (Cryo-EM) images. However, these techniques often fall short when predicting the structures of protein complexes (PCs), which involve multiple proteins. In our study, we investigate using atomic force microscopy (AFM) combined with deep learning to predict the 3D structures of PCs. AFM generates height maps that depict the PCs in various random orientations, providing a rich information for training a neural network to predict the 3D structures. We then employ the pre-trained UpFusion model (which utilizes a conditional diffusion model for synthesizing novel views) to train an instance-specific NeRF model for 3D reconstruction. The performance of UpFusion is evaluated through zero-shot predictions of 3D protein structures using AFM images. The challenge, however, lies in the time-intensive and impractical nature of collecting actual AFM images. To address this, we use a virtual AFM imaging process that transforms a `PDB' protein file into multi-view 2D virtual AFM images via volume rendering techniques. We extensively validate the UpFusion architecture using both virtual and actual multi-view AFM images. Our results include a comparison of structures predicted with varying numbers of views and different sets of views. This novel approach holds significant potential for enhancing the accuracy of protein complex structure predictions with further fine-tuning of the UpFusion network.

Figures (6)

• Figure 1: AlphaFold2 multimer (right) is unable to accurately predict the structure of the WRC protein complex (left).
• Figure 2: UpFusion architecture for synthesizing novel views from unposed input images.
• Figure 3: Virtual AFM: GPU-accelerated volume rendering of 3D voxelized protein structure to generate AFM-like virtual 2D images.
• Figure 4: Comparing the predictions corresponding to different number of input views.
• Figure 5: Comparing different set of input views for same protein structure, prediction 1 and 2 corresponds to input set 1 and 2, respectively.