Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Speak to a Protein: An Interactive Multimodal Co-Scientist for Protein Analysis

Carles Navarro, Mariona Torrens, Philipp Thölke, Stefan Doerr, Gianni De Fabritiis

TL;DR

Speak to a Protein presents an interactive AI co-scientist that grounds protein analysis in a synchronized multimodal interface combining natural language, a live 3D visualization scene, and executable Python code. By tightly integrating literature retrieval (PubMed Central via PMCIDs), UniProt enrichment, ChEMBL bioactivity data, and PDB structural data within a browser-based sandbox, the system enables real-time, evidence-based questioning from concept to visualization. Case studies on D3R and CDK2 demonstrate end-to-end workflows: from structure identification and pocket analysis to SAR interpretation and automated, publication-ready reporting. The approach reduces the time from initial questions to evidence, lowers barriers to advanced structural analyses, and facilitates hypothesis generation by unifying language, data, and 3D models in a single co-scientist framework.

Abstract

Building a working mental model of a protein typically requires weeks of reading, cross-referencing crystal and predicted structures, and inspecting ligand complexes, an effort that is slow, unevenly accessible, and often requires specialized computational skills. We introduce \emph{Speak to a Protein}, a new capability that turns protein analysis into an interactive, multimodal dialogue with an expert co-scientist. The AI system retrieves and synthesizes relevant literature, structures, and ligand data; grounds answers in a live 3D scene; and can highlight, annotate, manipulate and see the visualization. It also generates and runs code when needed, explaining results in both text and graphics. We demonstrate these capabilities on relevant proteins, posing questions about binding pockets, conformational changes, or structure-activity relationships to test ideas in real-time. \emph{Speak to a Protein} reduces the time from question to evidence, lowers the barrier to advanced structural analysis, and enables hypothesis generation by tightly coupling language, code, and 3D structures. \emph{Speak to a Protein} is freely accessible at https://open.playmolecule.org.

Speak to a Protein: An Interactive Multimodal Co-Scientist for Protein Analysis

TL;DR

Speak to a Protein presents an interactive AI co-scientist that grounds protein analysis in a synchronized multimodal interface combining natural language, a live 3D visualization scene, and executable Python code. By tightly integrating literature retrieval (PubMed Central via PMCIDs), UniProt enrichment, ChEMBL bioactivity data, and PDB structural data within a browser-based sandbox, the system enables real-time, evidence-based questioning from concept to visualization. Case studies on D3R and CDK2 demonstrate end-to-end workflows: from structure identification and pocket analysis to SAR interpretation and automated, publication-ready reporting. The approach reduces the time from initial questions to evidence, lowers barriers to advanced structural analyses, and facilitates hypothesis generation by unifying language, data, and 3D models in a single co-scientist framework.

Abstract

Building a working mental model of a protein typically requires weeks of reading, cross-referencing crystal and predicted structures, and inspecting ligand complexes, an effort that is slow, unevenly accessible, and often requires specialized computational skills. We introduce \emph{Speak to a Protein}, a new capability that turns protein analysis into an interactive, multimodal dialogue with an expert co-scientist. The AI system retrieves and synthesizes relevant literature, structures, and ligand data; grounds answers in a live 3D scene; and can highlight, annotate, manipulate and see the visualization. It also generates and runs code when needed, explaining results in both text and graphics. We demonstrate these capabilities on relevant proteins, posing questions about binding pockets, conformational changes, or structure-activity relationships to test ideas in real-time. \emph{Speak to a Protein} reduces the time from question to evidence, lowers the barrier to advanced structural analysis, and enables hypothesis generation by tightly coupling language, code, and 3D structures. \emph{Speak to a Protein} is freely accessible at https://open.playmolecule.org.
Paper Structure (18 sections, 4 figures, 4 tables)

This paper contains 18 sections, 4 figures, 4 tables.

Table of Contents

  1. Keywords
  2. Introduction
  3. Related Work
  4. System Overview: Speak to a Protein
  5. Architecture
  6. Literature Search and Comprehension
  7. UniProt: Accession Discovery and Rich Entry Information
  8. ChEMBL: Bioactivity and SAR Tables
  9. Structural Repositories: PDB and Predicted Models
  10. Multimodal Grounding and Interfaces with Scientists
  11. Experiments
  12. Speaking about the Dopamine D3 receptor (D3R)
  13. Speaking about the Cyclin-dependent kinase 2 (CDK2)
  14. Automated Report Generation.
  15. Conclusion and Limitations
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Overview of the system architecture. The system consists of a frontend with a protein viewer and chat interface, which includes a virtual file system and Python sandbox for automated code execution and viewer manipulation. The LLM Agent is the main orchestrator that interacts with the user and calls a set of custom tools through Model Context Protocol (MCP). These tools include a literature search for a given protein and text query, retrieving specialized data through APIs such as UniProt, PDBe and ChEMBL, as well as executing Python code in a sandbox environment with a dedicated virtual file system.
  • Figure 2: Interactive analysis of the CDK2 structure (PDB: 1AQ1) using the Speak to a Protein multimodal assistant. The user enters natural language queries in the AI chat panel (right), and the system responds with both textual answers and real-time updates to the 3D visualization (left). Python code executed in the integrated sandbox is shown in the console (bottom), providing full transparency and reproducibility of the underlying analyses.
  • Figure 3: Retrieval and exploration of potent D3R inhibitors. Example user query and system workflow for fetching all known D3R inhibitors and their affinities using Speak to a Protein. The chat panel shows the AI’s reasoning, including the tools used and the final response. On the left, the interface displays an interactive table of D3R inhibitors, including chemical structures, ChEMBL IDs, assay details, and more, enabling direct exploration and further analysis.
  • Figure 4: CDK2 structure–activity analysis. (a) Data integration and bioactivity analysis derived from ChEMBL datasets, demonstrating seamless integration of structural and bioactivity data through natural language queries. (b) Structural alignment of the top 20 most potent CDK2-ligand complexes with focused visualization of ATP-binding pockets.