Multimodal Model for Computational Pathology:Representation Learning and Image Compression

Abstract

Whole slide imaging (WSI) has transformed digital pathology by enabling computational analysis of gigapixel histopathology images. Recent foundation model advances have accelerated progress in computational pathology, facilitating joint reasoning across pathology images, clinical reports, and structured data. Despite this progress, challenges remain: the extreme resolution of WSIs creates computational hurdles for visual learning; limited expert annotations constrain supervised approaches; integrating multimodal information while preserving biological interpretability remains difficult; and the opacity of modeling ultra-long visual sequences hinders clinical transparency. This review comprehensively surveys recent advances in multimodal computational pathology. We systematically analyze four research directions: (1) self-supervised representation learning and structure-aware token compression for WSIs; (2) multimodal data generation and augmentation; (3) parameter-efficient adaptation and reasoning-enhanced few-shot learning; and (4) multi-agent collaborative reasoning for trustworthy diagnosis. We specifically examine how token compression enables cross-scale modeling and how multi-agent mechanisms simulate a pathologist's "Chain of Thought" across magnifications to achieve uncertainty-aware evidence fusion. Finally, we discuss open challenges and argue that future progress depends on unified multimodal frameworks integrating high-resolution visual data with clinical and biomedical knowledge to support interpretable and safe AI-assisted diagnosis.

Figures (6)

• Figure 1: Timeline of milestone AI models in computational pathology (2021–2025). The chart illustrates the chronological progression of key literature reviewed in this survey. Models are positioned according to their publication or preprint release dates, capturing the rapid evolution from foundation models to recent advanced multimodal frameworks.
• Figure 2: Schematic illustration of the data scale difference between pathological whole slide images and natural images. WSIs typically contain billions of pixels, imposing extremely high computational and storage demands. Moreover, due to tissue heterogeneity and uneven lesion distribution, diagnostically critical regions are often sparsely scattered within the vast background.
• Figure 3: Schematic of a multi-task self-supervised learning framework for pathological images. The framework jointly optimizes three pretext tasks: (a) masked image modeling to reconstruct masked patches, enabling the model to learn tissue structure and context; (b) instance-level contrastive learning that pulls augmented views of the same image together while pushing different views apart, capturing discriminative morphological features; and (c) cross-resolution consistency learning to enforce feature invariance across magnifications. Combined pretraining yields a general-purpose pathology foundation model with strong diagnostic discriminability.
• Figure 4: Schematic diagram of adaptive multi-resolution context modeling for whole slide images. The framework constructs an image pyramid across magnifications (e.g., 5×, 10×, 20×, 40×) as multi-scale inputs. A hierarchical multi-resolution visual encoder captures global tissue context at low resolutions and fine cellular details at high resolutions. Cross-scale attention modules enable bidirectional information flow, where global tokens guide local detail extraction and local features refine global representations, building a complete context pyramid from cellular to tissue level.
• Figure 5: Schematic illustration of few-shot adaptation based on multimodal foundation models. Given a small support set of labeled pathology images and corresponding clinical descriptions, a pretrained multimodal model is adapted to new diagnostic tasks using parameter-efficient fine-tuning techniques (e.g., adapters, prompt tuning, or low-rank adaptation). The model leverages prior knowledge from large-scale pretraining to rapidly generalize to novel disease subtypes with only a few examples, significantly reducing the need for extensive expert annotations.