PathoLM: Identifying pathogenicity from the DNA sequence through the Genome Foundation Model

TL;DR

PathoLM leverages a genome foundation model based on the Nucleotide Transformer v2 to identify pathogenicity directly from DNA sequences, addressing limitations of alignment-based methods and traditional ML in detecting novel pathogens. Using a diverse, multi-species pretraining corpus and k-mer tokenization, PathoLM achieves strong zero-shot and few-shot performance for binary pathogenicity and superior ESKAPEE species classification with PathoLM-Sp. The approach demonstrates robust performance across varying sequence lengths and dataset partitions, outperforming state-of-the-art methods like DciPatho and traditional ML baselines, while highlighting practical considerations such as context length and computational demands. The work provides a valuable dataset and framework for rapid pathogen surveillance and informs future improvements in genome-based pathogen detection with foundation models.

Abstract

Pathogen identification is pivotal in diagnosing, treating, and preventing diseases, crucial for controlling infections and safeguarding public health. Traditional alignment-based methods, though widely used, are computationally intense and reliant on extensive reference databases, often failing to detect novel pathogens due to their low sensitivity and specificity. Similarly, conventional machine learning techniques, while promising, require large annotated datasets and extensive feature engineering and are prone to overfitting. Addressing these challenges, we introduce PathoLM, a cutting-edge pathogen language model optimized for the identification of pathogenicity in bacterial and viral sequences. Leveraging the strengths of pre-trained DNA models such as the Nucleotide Transformer, PathoLM requires minimal data for fine-tuning, thereby enhancing pathogen detection capabilities. It effectively captures a broader genomic context, significantly improving the identification of novel and divergent pathogens. We developed a comprehensive data set comprising approximately 30 species of viruses and bacteria, including ESKAPEE pathogens, seven notably virulent bacterial strains resistant to antibiotics. Additionally, we curated a species classification dataset centered specifically on the ESKAPEE group. In comparative assessments, PathoLM dramatically outperforms existing models like DciPatho, demonstrating robust zero-shot and few-shot capabilities. Furthermore, we expanded PathoLM-Sp for ESKAPEE species classification, where it showed superior performance compared to other advanced deep learning methods, despite the complexities of the task.

Figures (3)

• Figure 1: Model Architecture of PathoLM. (A) Pretraining steps of the genome foundation model on a large-scale dataset, where tokenization and masking are applied before feeding the input sequence into the transformer block. (B) Finetuning steps for the pathogen identification task using specific datasets, where no masking is applied.
• Figure 2: A: Performance comparison between PathoLM and state-of-the-art method DciPatho on the binary pathogen prediction task in terms of Accuracy, F1-score, AUC-ROC, MCC. B: Comparison of F1-scores for various genome language models, highlighting their performance in zero-shot and few-shot scenarios.
• Figure 3: A: Performance comparison between PathoLM-Sp and machine learning Random Forest, XGBoost and deep learning methods CNN, LSTM on the ESKAPEE species classification task. PathoLM-Sp outperforms all other methods across multiple evaluation metrics. B: Performance comparison of PathoLM-Sp and other methods for varied sequence length from 150 bp to 50k bp. PathoLM-Sp consistently performs well for all sequence lengths. C: Performance comparison of PathoLM for different train and test set data created using mmseq2 sequence clustering. The similiarity-threshold and coverage 80, 60, 40 are shown as mmseq80, mmse60, mmseq40.