SMILES-Mamba: Chemical Mamba Foundation Models for Drug ADMET Prediction

TL;DR

SMILES-Mamba tackles ADMET prediction by combining self-supervised pretraining on unlabeled SMILES with task-specific fine-tuning. It uses a Mamba (S4-based) backbone to capture long-range chemical dependencies, pretraining on $ZINC$ 250K and fine-tuning on 22 ADMET tasks. It outperforms several baselines, achieving the highest scores in 14 tasks and strong performance across others, highlighting the effectiveness of self-supervised learning for molecular property prediction and reducing reliance on labeled data. The work suggests promising directions for accelerating drug discovery via unlabeled data and modality-specific representations.

Abstract

In drug discovery, predicting the absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties of small-molecule drugs is critical for ensuring safety and efficacy. However, the process of accurately predicting these properties is often resource-intensive and requires extensive experimental data. To address this challenge, we propose SMILES-Mamba, a two-stage model that leverages both unlabeled and labeled data through a combination of self-supervised pretraining and fine-tuning strategies. The model first pre-trains on a large corpus of unlabeled SMILES strings to capture the underlying chemical structure and relationships, before being fine-tuned on smaller, labeled datasets specific to ADMET tasks. Our results demonstrate that SMILES-Mamba exhibits competitive performance across 22 ADMET datasets, achieving the highest score in 14 tasks, highlighting the potential of self-supervised learning in improving molecular property prediction. This approach not only enhances prediction accuracy but also reduces the dependence on large, labeled datasets, offering a promising direction for future research in drug discovery.

Figures (1)

• Figure 1: Some examples of SMILES strings.