RIDER: 3D RNA Inverse Design with Reinforcement Learning-Guided Diffusion

TL;DR

RIDER addresses the limitation of optimizing RNA inverse design with native sequence recovery by directly targeting 3D structural fidelity. It introduces a two-stage pipeline: (i) RIDE, a structure-conditioned diffusion model pre-trained with NSR as supervision, and (ii) RIDER, reinforcement-learning fine-tuning that optimizes for 3D self-consistency using multiple metrics. The approach yields substantial gains in 3D structural similarity (over $100\%$ across metrics) and discovers high-quality designs beyond native sequences, with demonstrated generalization across predictors (e.g., AlphaFold3) and reward designs. This framework advances practical RNA design by aligning optimization objectives with true structural fidelity, potentially accelerating development of RNA therapeutics and synthetic biology tools.

Abstract

The inverse design of RNA three-dimensional (3D) structures is crucial for engineering functional RNAs in synthetic biology and therapeutics. While recent deep learning approaches have advanced this field, they are typically optimized and evaluated using native sequence recovery, which is a limited surrogate for structural fidelity, since different sequences can fold into similar 3D structures and high recovery does not necessarily indicate correct folding. To address this limitation, we propose RIDER, an RNA Inverse DEsign framework with Reinforcement learning that directly optimizes for 3D structural similarity. First, we develop and pre-train a GNN-based generative diffusion model conditioned on the target 3D structure, achieving a 9% improvement in native sequence recovery over state-of-the-art methods. Then, we fine-tune the model with an improved policy gradient algorithm using four task-specific reward functions based on 3D self-consistency metrics. Experimental results show that RIDER improves structural similarity by over 100% across all metrics and discovers designs that are distinct from native sequences.

Figures (17)

• Figure 1: Visualization of sequences (a), (b), and (c), and their corresponding 3D structures ($\alpha$), ($\beta$), and ($\gamma$) predicted by RhoFold shen2024accurate. Although sequences (a) and (b) differ by only 3 nucleotides, and (b) and (c) by 5 nucleotides, their folded structures exhibit clear differences.
• Figure 2: Overview of the RIDER framework. RNA tertiary structures are processed by a GVP-GNN encoder to produce structural embeddings. These embeddings condition the diffusion model for sequence generation, which is further optimized by RL to maximize structural similarity.
• Figure 3: Results of supervised learning pre-training. A. Comparison of native sequence recovery on the test set. RIDE is compared against RiboDiffusion and gRNAde. The best NSR among $16$ designs per target (sampled at temperature $0.1$) is reported. B. Relationship between NSR and structural similarity (GDT_TS and RMSD) for RIDE designs. Color denotes RMSD. C. Correlation among GDT_TS, TM-score, and RMSD for the designed structures.
• Figure 4: Results of reinforcement learning fine-tuning. A. GDT_TS comparison on $14$ RNA structures of interest das2010atomic for gRNAde, RIDE (pre-trained), and RIDER (fine-tuned with $R^{\texttt{gdt\_rmsd}}$). B. Comparison of native sequence recovery before (RIDE) and after (RIDER) RL fine-tuning. Color indicates GDT_TS after RL fine-tuning. The results for the other two metrics are provided in Appendix \ref{['sec:additional']}.
• Figure 5: Visualization of designed examples. Structures folded from sequences generated by RIDER are shown in color, while the target structures are shown in semi-transparent yellow.