Towards a Better Model with Dual Transformer for Drug Response Prediction

TL;DR

The decoupled dual transformer structure with edge embedded for drug respond prediction (TransEDRP), which is used for the representation of cell line genomics and drug respectively, is proposed, which is better than the current mainstream approach in all evaluation indicators.

Abstract

GNN-based methods have achieved excellent results as a mainstream task in drug response prediction tasks in recent years. Traditional GNN methods use only the atoms in a drug molecule as nodes to obtain the representation of the molecular graph through node information passing, whereas the method using the transformer can only extract information about the nodes. However, the covalent bonding and chirality of a drug molecule have a great influence on the pharmacological properties of the molecule, and these information are implied in the chemical bonds formed by the edges between the atoms. In addition, CNN methods for modelling cell lines genomics sequences can only perceive local rather than global information about the sequence. In order to solve the above problems, we propose the decoupled dual transformer structure with edge embedded for drug respond prediction (TransEDRP), which is used for the representation of cell line genomics and drug respectively. For the drug branch, we encoded the chemical bond information within the molecule as the embedding of the edge in the molecular graph, extracted the global structural and biochemical information of the drug molecule using graph transformer. For the branch of cell lines genomics, we use the multi-headed attention mechanism to globally represent the genomics sequence. Finally, the drug and genomics branches are fused to predict IC50 values through the transformer layer and the fully connected layer, which two branches are different modalities. Extensive experiments have shown that our method is better than the current mainstream approach in all evaluation indicators.

Figures (7)

• Figure 1: Fig. (a) shows a pair of chiral isomers as an example. Fig. (b) shows the edge features including covalent bonds, aromatic rings, etc., along with the chirality of the atoms.
• Figure 2: The gene expression sequence of CAL-29, which is derived from transitional cell carcinoma of the bladder. The perceived fields on the sequence using 1D-CNN and the Attention layer are also compared.
• Figure 3: Illustration of the TransEDRP framework, which consists of three components: the graph transformer with edge embedding module, the transformer-based genomic encoder module, and the multi-modal soft fusion module.
• Figure 4: The figure shows the response prediction results for 15 classical drugs on GDSCv2 by mainstream methods, where $Pearson$ is chosen as the evaluation metric.
• Figure 5: A pair of chiral isomers, which are mirror-symmetric and cannot be overlapped.