TICON: A Slide-Level Tile Contextualizer for Histopathology Representation Learning

TL;DR

This paper addresses the need for meaningful tile context in histopathology by introducing TICON, a universal tile contextualizer that can ingest embeddings from any tile encoder. Using Omni-Feature Masked Modeling, TICON trains a single shared slide-level transformer to contextualize tile embeddings, aligning diverse encoders through a multi-target cross-decoder objective. The approach yields state-of-the-art results on tile-level benchmarks (HEST-Bench, THUNDER, CATCH) and slide-level benchmarks (Patho-Bench), and demonstrates data-efficient pretraining, with a slide-level model trained on just 11K WSIs outperforming methods using far more data. Moreover, TICON supports efficient adaptation to unseen tile encoders via lightweight projections, and when coupled with a TANGLE-based MIL pretraining, establishes a competitive, data-efficient slide-level foundation model, highlighting the practical impact of unified, context-rich tile representations in computational pathology.

Abstract

The interpretation of small tiles in large whole slide images (WSI) often needs a larger image context. We introduce TICON, a transformer-based tile representation contextualizer that produces rich, contextualized embeddings for ''any'' application in computational pathology. Standard tile encoder-based pipelines, which extract embeddings of tiles stripped from their context, fail to model the rich slide-level information essential for both local and global tasks. Furthermore, different tile-encoders excel at different downstream tasks. Therefore, a unified model is needed to contextualize embeddings derived from ''any'' tile-level foundation model. TICON addresses this need with a single, shared encoder, pretrained using a masked modeling objective to simultaneously unify and contextualize representations from diverse tile-level pathology foundation models. Our experiments demonstrate that TICON-contextualized embeddings significantly improve performance across many different tasks, establishing new state-of-the-art results on tile-level benchmarks (i.e., HEST-Bench, THUNDER, CATCH) and slide-level benchmarks (i.e., Patho-Bench). Finally, we pretrain an aggregator on TICON to form a slide-level foundation model, using only 11K WSIs, outperforming SoTA slide-level foundation models pretrained with up to 350K WSIs.

Figures (4)

• Figure 1: Tile classification in a skin cancer WSI from the CATCH dataset catch. Prediction without context leads to incorrect predictions (Mast Cell Tumor instead of Histiocytoma). Results with our contextualizer, TICON, are much closer to the ground truth.
• Figure 2: TICON: An Omni Tile Contextualizer that can contextualize embeddings from any tile encoder. (---) represent input projectors for tile encoders used in pretraining. (- -) represent input projectors used in adapting TICON to new tile encoders.
• Figure 3: Overview of the pretraining framework. (Left) Grid sampling and tile embedding extraction using a set of tile encoders ($\phi_1, \phi_2, \dots, \phi_T$). (Right) An input tile encoder ($\phi_i$) is sampled randomly at each iteration, and its embeddings are masked. The remaining visible embeddings are passed through a $\phi_i$-specific input projector ($\rho_i$) and then a shared encoder. A shared decoder, paired with output projectors specific to each tile encoder, then reconstructs the masked embeddings corresponding to all tile encoders ($\phi_1, \dots, \phi_T$).
• Figure 4: Visualization of tile classification results on a WSI from the CATCH dataset. The left panel (baseline) shows classification using non-contextual tile embeddings, whereas the right panel (TICON) displays classification with contextualized embeddings. TICON produces less noisy predictions and corrects many local misclassifications (green boxes). However, we also observe shared failure modes (orange box) where both methods misclassify a region. This suggests limitations in the underlying tile encoder's features. Indeed, the latter can miss some necessary information that even contextualization cannot retrieve.