Quantifying Cross-Attention Interaction in Transformers for Interpreting TCR-pMHC Binding

TL;DR

This work tackles the interpretability challenge of encoder-decoder transformers in TCR-pMHC binding prediction by introducing Quantifying Cross-Attention Interaction (QCAI). QCAI deconstructs cross-attention into query- and key-input attributions using a GradCAM-style framework and a Moore-Penrose pseudoinverse-based separation, then aggregates explanations across layers. To evaluate explanations rigorously, the authors construct TCR-XAI, a 274-structure benchmark with residue-level ground truth distances, and introduce BRHR as a binding-region accuracy metric. On a state-of-the-art TCR-pMHC model (TULIP), QCAI achieves state-of-the-art interpretability and competitive predictive performance, outperforming encoder-only baselines on multiple metrics (ROC-AUC, LOdds, AOPC, BRHR). The method holds promise for broader application to cross-attention in diverse transformer models and protein–protein interaction tasks.

Abstract

CD8+ "killer" T cells and CD4+ "helper" T cells play a central role in the adaptive immune system by recognizing antigens presented by Major Histocompatibility Complex (pMHC) molecules via T Cell Receptors (TCRs). Modeling binding between T cells and the pMHC complex is fundamental to understanding basic mechanisms of human immune response as well as in developing therapies. While transformer-based models such as TULIP have achieved impressive performance in this domain, their black-box nature precludes interpretability and thus limits a deeper mechanistic understanding of T cell response. Most existing post-hoc explainable AI (XAI) methods are confined to encoder-only, co-attention, or model-specific architectures and cannot handle encoder-decoder transformers used in TCR-pMHC modeling. To address this gap, we propose Quantifying Cross-Attention Interaction (QCAI), a new post-hoc method designed to interpret the cross-attention mechanisms in transformer decoders. Quantitative evaluation is a challenge for XAI methods; we have compiled TCR-XAI, a benchmark consisting of 274 experimentally determined TCR-pMHC structures to serve as ground truth for binding. Using these structures we compute physical distances between relevant amino acid residues in the TCR-pMHC interaction region and evaluate how well our method and others estimate the importance of residues in this region across the dataset. We show that QCAI achieves state-of-the-art performance on both interpretability and prediction accuracy under the TCR-XAI benchmark.

Figures (15)

• Figure 1: Quantifying Cross-Attention Interaction (QCAI) is a post-hoc explanation method designed for cross-attention mechanisms. In this paper, we show that QCAI enables insight into the structural basis for TCR-pMHC binding.
• Figure 2: ROC-AUC of predicted importance scores for TCR-pMHC binding site identification across a threshold of interaction distances demonstrates that QCAI surpasses competing methods in all cases.
• Figure 3: Comparison of TCR-pMHC Binding Region Hit Rate (BRHR) across different methods on different chains. At any selected percentile of distance/importance, the higher the hit rate the more closely the importance tracks physical interaction distance. QCAI surpasses other methods in all practical cases.
• Figure 4: Case studies on systems from TCR-XAI. (a) We consider the same TCR-pMHC bound in two distinct binding orientations. For this system QCAI identifies key residues from both orientations. (b) We consider the same pMHC bound to two distinct TCRs. Here QCAI identifies the importance of the hairpin region of CDR3a in both cases.
• Figure 5: Case studies of two closely related TCR-pMHC complexes from TCR-XAI. These complexes differ by only two amino acids in the CDR3b, highlighted in the figure with red rectangles.