Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RxnNano:Training Compact LLMs for Chemical Reaction and Retrosynthesis Prediction via Hierarchical Curriculum Learning

Ran Li, Shimin Di, Haowei LI, Luanshi Bu, Jiachuan Wang, Wangze Ni, Lei Chen

TL;DR

This work proposes a unified framework that prioritizes chemical understanding over scale through three key innovations: a {Latent Chemical Consistency} objective that models reactions as movements on a continuous chemical manifold, ensuring reversible and physically plausible transformations.

Abstract

Chemical reaction prediction is pivotal for accelerating drug discovery and synthesis planning. Despite advances in data-driven models, current approaches are hindered by an overemphasis on parameter and dataset scaling. Some methods coupled with evaluation techniques that bypass fundamental challenges in reaction representation and fail to capture deep chemical intuition like reaction common sense and {topological atom mapping logic}. We argue that the core challenge lies in instilling these knowledge into the models. To this end, we propose a unified framework that prioritizes chemical understanding over scale through three key innovations: (1) a {Latent Chemical Consistency} objective that models reactions as movements on a continuous chemical manifold, ensuring reversible and physically plausible transformations; (2) a {Hierarchical Cognitive Curriculum} that trains the model through progressive stages, from syntax mastery to semantic reasoning, building robust chemical intuition; (3) {Atom-Map Permutation Invariance (AMPI)}, which force the model to learn invariant relational topology and balance multi-task learning. (4)and structured plan-based reasoning to improve the performance of the LLMs. Our compact {0.5B-parameter model}, \textbf{RxnNano} significantly outperforms fine-tuned LLMs ten times larger (>7B) and all the domain baselines, achieving a 23.5\% Top-1 accuracy improvement on rigorous benchmarks without test-time augmentation. https://github.com/rlisml/RxnNano.

RxnNano:Training Compact LLMs for Chemical Reaction and Retrosynthesis Prediction via Hierarchical Curriculum Learning

TL;DR

This work proposes a unified framework that prioritizes chemical understanding over scale through three key innovations: a {Latent Chemical Consistency} objective that models reactions as movements on a continuous chemical manifold, ensuring reversible and physically plausible transformations.

Abstract

Chemical reaction prediction is pivotal for accelerating drug discovery and synthesis planning. Despite advances in data-driven models, current approaches are hindered by an overemphasis on parameter and dataset scaling. Some methods coupled with evaluation techniques that bypass fundamental challenges in reaction representation and fail to capture deep chemical intuition like reaction common sense and {topological atom mapping logic}. We argue that the core challenge lies in instilling these knowledge into the models. To this end, we propose a unified framework that prioritizes chemical understanding over scale through three key innovations: (1) a {Latent Chemical Consistency} objective that models reactions as movements on a continuous chemical manifold, ensuring reversible and physically plausible transformations; (2) a {Hierarchical Cognitive Curriculum} that trains the model through progressive stages, from syntax mastery to semantic reasoning, building robust chemical intuition; (3) {Atom-Map Permutation Invariance (AMPI)}, which force the model to learn invariant relational topology and balance multi-task learning. (4)and structured plan-based reasoning to improve the performance of the LLMs. Our compact {0.5B-parameter model}, \textbf{RxnNano} significantly outperforms fine-tuned LLMs ten times larger (>7B) and all the domain baselines, achieving a 23.5\% Top-1 accuracy improvement on rigorous benchmarks without test-time augmentation. https://github.com/rlisml/RxnNano.
Paper Structure (30 sections, 1 theorem, 12 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 30 sections, 1 theorem, 12 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Molecular Representations and Chemical Encoding
  4. Methodological Paradigms
  5. Further Techniques for Performance Enhancement
  6. Methodology
  7. Overview
  8. Reaction Representation
  9. Problem Formalization
  10. Hierarchical Cognitive Curriculum
  11. Stage 1: Syntactic Phase
  12. Stage 2: Denoising Phase
  13. Stage 3: Semantic Phase with AMPI
  14. Parameter-Efficient Fine-Tuning
  15. Plan-Based Reasoning as Latent Variable Model
  16. ...and 15 more sections

Key Result

Proposition 3.1

Let $\mathcal{R}$ be the space of chemical reactions and $\mathcal{Z}$ the space of valid mechanistic plans. If there exists a surjective mapping $\phi: \mathcal{Z} \rightarrow \mathcal{R}$ such that reactions with similar mechanisms have nearby representations in $\mathcal{Z}$, then the plan-based

Figures (4)

  • Figure 1: Comparison of ours and baselines
  • Figure 2: Overview of our framework: The Hierarchical Curriculum from syntax to semantics. The Cycle-Consistency ensuring topological robustness. The Relational Invariance mechanism for unbiased chemical reasoning and Plan-based token to enhance the inference.
  • Figure 3: Reaction types in USPTO-50K (left) and corresponding model performance metrics (right)
  • Figure 4: Top-5 accuracy v.s temperature.

Theorems & Definitions (4)

  • Definition 3.1: Reaction Prediction Tasks
  • Definition 3.2: Chemical Cycle-Consistency
  • Definition 3.3: AMPI Transformation
  • Proposition 3.1: Plan-Guided Generalization