Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification

Dexuan Ding, Ciyuan Peng, Endrowednes Kuantama, Jingcai Guo, Jia Wu, Jian Yang, Amin Beheshti, Ming-Hsuan Yang, Yuankai Qi

TL;DR

MVSC addresses the challenge of decoding high-dimensional 3D sMRI for Alzheimer's disease classification by learning a compact 2D visual surrogate aligned with frozen 2D foundation models. It introduces a Volume Context Encoder that leverages text-guided global context and an Adaptive Slice Fusion module for patch-level cross-slice integration, enabling efficient cross-slice dependencies without full 3D architectures. Across three large benchmarks (AIBL, OASIS-3, ADNI), MVSC with frozen 2D backbones achieves state-of-the-art or competitive AUC and macro-AUC in binary and multi-class settings, while maintaining a lightweight footprint. The approach demonstrates the feasibility of task-adaptive, multimodal surrogate learning to improve generalization across protocols and scanners.

Abstract

High-dimensional structural MRI (sMRI) images are widely used for Alzheimer's Disease (AD) diagnosis. Most existing methods for sMRI representation learning rely on 3D architectures (e.g., 3D CNNs), slice-wise feature extraction with late aggregation, or apply training-free feature extractions using 2D foundation models (e.g., DINO). However, these three paradigms suffer from high computational cost, loss of cross-slice relations, and limited ability to extract discriminative features, respectively. To address these challenges, we propose Multimodal Visual Surrogate Compression (MVSC). It learns to compress and adapt large 3D sMRI volumes into compact 2D features, termed as visual surrogates, which are better aligned with frozen 2D foundation models to extract powerful representations for final AD classification. MVSC has two key components: a Volume Context Encoder that captures global cross-slice context under textual guidance, and an Adaptive Slice Fusion module that aggregates slice-level information in a text-enhanced, patch-wise manner. Extensive experiments on three large-scale Alzheimer's disease benchmarks demonstrate our MVSC performs favourably on both binary and multi-class classification tasks compared against state-of-the-art methods.

Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification

TL;DR

MVSC addresses the challenge of decoding high-dimensional 3D sMRI for Alzheimer's disease classification by learning a compact 2D visual surrogate aligned with frozen 2D foundation models. It introduces a Volume Context Encoder that leverages text-guided global context and an Adaptive Slice Fusion module for patch-level cross-slice integration, enabling efficient cross-slice dependencies without full 3D architectures. Across three large benchmarks (AIBL, OASIS-3, ADNI), MVSC with frozen 2D backbones achieves state-of-the-art or competitive AUC and macro-AUC in binary and multi-class settings, while maintaining a lightweight footprint. The approach demonstrates the feasibility of task-adaptive, multimodal surrogate learning to improve generalization across protocols and scanners.

Abstract

High-dimensional structural MRI (sMRI) images are widely used for Alzheimer's Disease (AD) diagnosis. Most existing methods for sMRI representation learning rely on 3D architectures (e.g., 3D CNNs), slice-wise feature extraction with late aggregation, or apply training-free feature extractions using 2D foundation models (e.g., DINO). However, these three paradigms suffer from high computational cost, loss of cross-slice relations, and limited ability to extract discriminative features, respectively. To address these challenges, we propose Multimodal Visual Surrogate Compression (MVSC). It learns to compress and adapt large 3D sMRI volumes into compact 2D features, termed as visual surrogates, which are better aligned with frozen 2D foundation models to extract powerful representations for final AD classification. MVSC has two key components: a Volume Context Encoder that captures global cross-slice context under textual guidance, and an Adaptive Slice Fusion module that aggregates slice-level information in a text-enhanced, patch-wise manner. Extensive experiments on three large-scale Alzheimer's disease benchmarks demonstrate our MVSC performs favourably on both binary and multi-class classification tasks compared against state-of-the-art methods.
Paper Structure (36 sections, 21 equations, 2 figures, 8 tables)

This paper contains 36 sections, 21 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Learning on sMRI.
  4. Alzheimer's Disease Diagnosis.
  5. Medical Foundation Model.
  6. Method
  7. Notations.
  8. Volume Context Encoder (VoCE)
  9. Global Feature Aggregation.
  10. Adaptive Slice Fusion (ASF)
  11. Patch-level Feature Augmentation.
  12. Patch-aligned Cross-slice Fusion.
  13. Classification
  14. Experiments
  15. Set Up
  16. ...and 21 more sections

Figures (2)

  • Figure 1: Main paradigms of representative sMRI feature extraction for Alzheimer’s disease classification. The fire icon indicates trainable modules and the snowflake icon indicates frozen modules. (a) Fully volumetric 3D models that directly process sMRI volumes (e.g., M3T DBLP:conf/cvpr/JangH22). (b) Slice-wise methods with trainable aggregation (e.g., AXIAL DBLP:journals/corr/abs-2407-02418). (c) Slice-wise training-free approaches that rely on generic 2D foundation models and random feature projection (e.g., RAPTOR DBLP:conf/icml/AnJLGYS25). (d) Our lightweight MVSC framework, which learns a visual surrogate to enable lightweight and effective feature extraction with generic 2D foundation models (e.g., DINOv3 DBLP:journals/corr/abs-2508-10104). The reported performance are achieved on the largest AD classficiation dataset ADNI.
  • Figure 2: Main architecture of our multimodal visual surrogate compression (MVSC) framework. MVSC consists of two key components: (i) Volume Context Encoder (VoCE) (Section \ref{['subsec:voce']}), which aggregates patch-level visual features across all slices to learn a global volume representation guided by volume-level text; and (ii) Adaptive Slice Fusion (ASF) (Section \ref{['subsec:asf']}), which performs patch-aligned cross-slice fusion by integrating patch-level visual features with slice-level semantic information and the global representation to construct a 2D visual surrogate.