Patch-based Intuitive Multimodal Prototypes Network (PIMPNet) for Alzheimer's Disease classification

TL;DR

This work addresses Alzheimer’s disease diagnosis from 3D sMRI by incorporating age alongside imaging through an interpretable multimodal prototype network, PIMPNet. It extends prior prototypical networks with a trainable age-prototypes layer and a sparse, nonnegative classifier, enabling joint reasoning over image and demographic information. While age prototypes can be learned, they do not consistently improve predictive performance over image-only models, though they offer a foundation for future multimodal prototype training and more expressive fusion strategies. Overall, PIMPNet advances interpretable diagnostics for AD by providing prototype-driven explanations that unify imaging and age data, with clear directions for refining age–image interactions.

Abstract

Volumetric neuroimaging examinations like structural Magnetic Resonance Imaging (sMRI) are routinely applied to support the clinical diagnosis of dementia like Alzheimer's Disease (AD). Neuroradiologists examine 3D sMRI to detect and monitor abnormalities in brain morphology due to AD, like global and/or local brain atrophy and shape alteration of characteristic structures. There is a strong research interest in developing diagnostic systems based on Deep Learning (DL) models to analyse sMRI for AD. However, anatomical information extracted from an sMRI examination needs to be interpreted together with patient's age to distinguish AD patterns from the regular alteration due to a normal ageing process. In this context, part-prototype neural networks integrate the computational advantages of DL in an interpretable-by-design architecture and showed promising results in medical imaging applications. We present PIMPNet, the first interpretable multimodal model for 3D images and demographics applied to the binary classification of AD from 3D sMRI and patient's age. Despite age prototypes do not improve predictive performance compared to the single modality model, this lays the foundation for future work in the direction of the model's design and multimodal prototype training process

Figures (1)

• Figure 1: PIMPNet architecture