TC-SSA: Token Compression via Semantic Slot Aggregation for Gigapixel Pathology Reasoning

TL;DR

The proposed TC-SSA (Token Compression via Semantic Slot Aggregation), a learnable token compression framework that aggregates patch features into a fixed number of semantic slots, suggests that learnable semantic aggregation provides an effective trade-off between efficiency and diagnostic performance for gigapixel pathology reasoning.

Abstract

The application of large vision-language models to computational pathology holds great promise for diagnostic assistants but faces a critical computational bottleneck: the gigapixel scale of Whole Slide Images (WSIs). A single WSI typically contains over 105 patches, creating sequence lengths that exceed the constraints of standard Transformer architectures. Existing solutions often resort to spatial sampling, which risks discarding diagnostically critical evidence. To address this, we propose TC-SSA (Token Compression via Semantic Slot Aggregation), a learnable token compression framework that aggregates patch features into a fixed number of semantic slots. A gated routing module assigns patches to slots using sparse Top-2 routing, followed by weighted aggregation, enabling global slide coverage under a strict token budget. The resulting representation retains diagnostically relevant information while reducing the number of visual tokens to 1.7% of the original sequence. On the SlideBench(TCGA), our model achieves 78.34% overall accuracy and 77.14% on the diagnosis subset, outperforming sampling-based baselines under comparable token budgets. The method also generalizes to MIL classification, reaching AUC of 95.83% on TCGA-BRCA, 98.27% on TCGA-NSCLC and 79.80% on PANDA. These results suggest that learnable semantic aggregation provides an effective trade-off between efficiency and diagnostic performance for gigapixel pathology reasoning.

Figures (4)

• Figure 1: Overview of the Token Compression via Semantic Slot Aggregation (TC-SSA) framework.(a) Whole flow: a feature encoder extracts a variable-length feature sequence ($N$) from partitioned gigapixel WSI patches. The TC-SSA module compresses these dense features into a fixed token budget before feeding them into the VLM for downstream diagnostic reasoning. (b) TC-SSA Module: a gate network processes input features $X \in \mathbb{R}^{B \times N \times D}$ to determine routing probabilities, followed by Top-K selection. The auxiliary semantic affinity clustering objective ($\mathcal{L}_{aux}$) regularizes this routing distribution. Routed patch features undergo weighted aggregation into $K$ distinct slots, yielding the final compressed representation $X' \in \mathbb{R}^{B \times K \times D}$. (c) Slot Architecture: the internal configuration of each independent slot.
• Figure 2: Using only 32 visual tokens, TC-SSA achieves 78.34% overall accuracy on SlideBench(TCGA) benchmark, yielding a 58$\times$ compression ratio compared to original patch features. SlideChat is reported as an uncompressed reference, while full-WSI inference is infeasible due to out-of-memory (OOM) constraints.
• Figure 3: Multi-dataset t-SNE of patch embeddings grouped by slot. Colored clusters correspond to different tissue semantics, showing slot assignment induces coherent, dataset-consistent grouping in embedding space.
• Figure 4: Ablation experiments of the number of semantic slots $K$ on TCGA-BRCA and TCGA-NSCLC. We set $K=32$ by default.