Nature Language Model: Deciphering the Language of Nature for Scientific Discovery

TL;DR

NatureLM introduces a scalable, sequence-based science foundation model trained on multi-domain data (text, SMILES, FASTA, and materials representations) to enable cross-domain generation and design for discovery tasks across chemistry, biology, and materials science. By combining a two-stage token integration strategy, instruction-tuned post-training, and inference acceleration, NatureLM achieves strong, cross-domain performance and demonstrates capabilities in molecule and protein design, RNA/gRNA generation, retrosynthesis, and materials design, with larger models consistently outperforming smaller variants. The work also explores reinforcement learning, domain-specific fine-tuning, and dedicated benchmarks (e.g., Matbench, USPTO-50K) to push cross-domain applicability, while acknowledging current limitations in natural-language capabilities and few-shot learning. Overall, NatureLM represents a significant step toward a unified, generalist foundation model for scientific discovery, enabling integrated design and optimization across multiple domains with potential practical impact in drug discovery, material design, and therapeutic nucleotides.

Abstract

Foundation models have revolutionized natural language processing and artificial intelligence, significantly enhancing how machines comprehend and generate human languages. Inspired by the success of these foundation models, researchers have developed foundation models for individual scientific domains, including small molecules, materials, proteins, DNA, RNA and even cells. However, these models are typically trained in isolation, lacking the ability to integrate across different scientific domains. Recognizing that entities within these domains can all be represented as sequences, which together form the "language of nature", we introduce Nature Language Model (NatureLM), a sequence-based science foundation model designed for scientific discovery. Pre-trained with data from multiple scientific domains, NatureLM offers a unified, versatile model that enables various applications including: (i) generating and optimizing small molecules, proteins, RNA, and materials using text instructions; (ii) cross-domain generation/design, such as protein-to-molecule and protein-to-RNA generation; and (iii) top performance across different domains, matching or surpassing state-of-the-art specialist models. NatureLM offers a promising generalist approach for various scientific tasks, including drug discovery (hit generation/optimization, ADMET optimization, synthesis), novel material design, and the development of therapeutic proteins or nucleotides. We have developed NatureLM models in different sizes (1 billion, 8 billion, and 46.7 billion parameters) and observed a clear improvement in performance as the model size increases.

Figures (42)

• Figure 1: NatureLM is a GPT-style generative model trained on a diverse range of data, including small molecule compounds, proteins, DNA, RNA, materials, and both general and scientific texts, amounting to a total of 143 billion tokens. It is built on existing large language models by integrating new vocabularies for scientific entities and jointly pre-training all components. After the pre-training, the model undergoes additional instruction tuning using millions of curated instructions from scientific fields. Options for reinforcement learning and dedicated fine-tuning are also available to boost performance on specific tasks. Users can engage with NatureLM through natural language inputs. The model excels in various domains, achieving top results in tasks such as retrosynthesis (Section \ref{['sec:retro']}), SMILES-to-IUPAC translation (Section \ref{['sec:smiles_iupac']}), protein generation (Section \ref{['sec_prot_generation']}) and material property prediction (Section \ref{['sec:dedicate_tune_matbench']}), often matching or exceeding the capabilities of state-of-the-art specialized models.
• Figure 2: The scaling effect in NatureLM is obvious. The chart depicts the overall ranking of models with varying sizes, where a better rank is represented by the "outsider" bar. The 8x7B model achieves top performance in 19 tasks, while the 8B model excels in 3 tasks. 18 categories exhibited performance improvements with increasing model size (i.e., 8x7B demonstrated the best performance, followed by 8B, and then 1B), highlighting the potential of large foundation models for scientific applications.
• Figure 3: Example data from each domain. The small molecule is Aspirin (PubChem CID: 2244) and visualized by RDKit greg_landrum_2025_14779836. The protein snapshot is from the PDB bank with ID 7CAM Wang2020-vw. The DNA structure is split into chain I and chain J from PDB 1KX5 Davey2002im and visualized by UCSF Chimera chimera2004software. The material snapshot is from the material project with ID mp-1960 osti_1194803.
• Figure 4: Distribution of the pre-training data, measured by the number of tokens of each category.
• Figure 5: Statistics of post-training data, measured by the number of sequences.