M2oE: Multimodal Collaborative Expert Peptide Model

TL;DR

By integrating sequence and spatial structural information, employing expert model and Cross-Attention Mechanism, the model’s capabilities are balanced and improved and Experimental results indicate that the M2oE model performs excellently in complex task predictions.

Abstract

Peptides are biomolecules comprised of amino acids that play an important role in our body. In recent years, peptides have received extensive attention in drug design and synthesis, and peptide prediction tasks help us better search for functional peptides. Typically, we use the primary sequence and structural information of peptides for model encoding. However, recent studies have focused more on single-modal information (structure or sequence) for prediction without multi-modal approaches. We found that single-modal models are not good at handling datasets with less information in that particular modality. Therefore, this paper proposes the M2oE multi-modal collaborative expert peptide model. Based on previous work, by integrating sequence and spatial structural information, employing expert model and Cross-Attention Mechanism, the model's capabilities are balanced and improved. Experimental results indicate that the M2oE model performs excellently in complex task predictions.

Figures (1)

• Figure 1: The framework of the M2oE. The model is structured with an encoding module and a decoding module, incorporating the interactive attention mechanism in the SCMoE module and MoE token allocation to enhance the comprehensive ability of the M2oE encoding model. Additionally, MoE optimization is achieved through auxiliary loss. The decoding module utilizes MLP and learnable parameters $\alpha$ from both modes for making predictions.