Entropy-Guided Dynamic Tokens for Graph-LLM Alignment in Molecular Understanding

TL;DR

The paper tackles the challenge of aligning molecular graphs with large language models by introducing EDT-Former, a connector that uses entropy-guided dynamic substructure tokens and a Dynamic Query Transformer to fuse frozen graph encoders with frozen LLMs. By replacing fixed-length modality tokens with data-driven, structure-aware patches and anchors, EDT-Former maintains substructure fidelity (including stereochemistry) while enabling efficient training that updates only the bridge (and embedding layer). The approach yields state-of-the-art results across MoleculeQA, Mol-Instructions, and MoleculeNet benchmarks, and it demonstrates strong generalization with reduced compute compared to full LLM fine-tuning. The work provides a scalable, interpretable, and reproducible recipe for multimodal molecular understanding and offers a framework adaptable to broader graph-domain tasks.

Abstract

Molecular understanding is central to advancing areas such as scientific discovery, yet Large Language Models (LLMs) struggle to understand molecular graphs effectively. Existing graph-LLM bridges often adapt the Q-Former-style connector with fixed-length static tokens, which is originally designed for vision tasks. These designs overlook stereochemistry and substructural context and typically require costly LLM-backbone fine-tuning, limiting efficiency and generalization. We introduce EDT-Former, an Entropy-guided Dynamic Token Transformer that generates tokens aligned with informative molecular patches, thereby preserving both local and global structural features for molecular graph understanding. Beyond prior approaches, EDT-Former enables alignment between frozen graph encoders and LLMs without tuning the LLM backbone (excluding the embedding layer), resulting in computationally efficient finetuning, and achieves stateof-the-art results on MoleculeQA, Molecule-oriented Mol-Instructions, and property prediction benchmarks (TDC, MoleculeNet), underscoring its effectiveness for scalable and generalizable multimodal molecular understanding

Figures (10)

• Figure 1: LLM joint fine-tuning efficiency and benchmark performance preview of EDT-Former.
• Figure 2: Illustration of motivation--Loss of structure. Comparison of molecules of different sizes (atom counts $N=16$ and $N=50$) encoded by the same fixed query length Q-Former bridge (8 query tokens) to Llama-3.1-8B backbone, with example prompts and generated responses.
• Figure 3: The architecture of EDT-Former. (a) Entropy-based Patching segments node embeddings into patches to produce dynamic query tokens. (b) EDT-Former integrates anchors and dynamic queries through Dynamic Query Transformer to align the molecular graph with the LLM.
• Figure 4: Illustration of entropy-guided patching on an example molecule. Atom-level entropy is plotted along the molecular sequence, and a new patch is initiated after each local maximum.
• Figure 5: Ablation study of components on the MoleculeQA dataset. Accuracy is reported across four task types (Structure, Source, Property, and Application) when removing each component (modality fusion, Entropy-Guided Patching, or Dynamic Query Transformer).

Theorems & Definitions (14)

• Lemma 1: Surprisal peaks mark change points
• proof : Sketch
• Lemma 2: Pooling loss is entropy-controlled
• Proposition 1: Peak cutting minimizes an upper bound on loss
• proof : Sketch
• Corollary 1: Adaptive token count matches structural complexity
• Lemma 3: Anchor-conditioned Lipschitz stability
• proof : Sketch
• Proposition 2: Connector-only can emulate low-rank input adaptation
• proof : Sketch