A Computer Vision Hybrid Approach: CNN and Transformer Models for Accurate Alzheimer's Detection from Brain MRI Scans

TL;DR

This work addresses the need for accurate, robust Alzheimer's disease classification from brain MRI by systematically comparing five CNN architectures, five Transformer models, and a novel hybrid Evan_V2. The approach combines local feature extraction from CNNs with global context modeling from Transformers through probability-level fusion in the Evan_V2 ensemble, achieving superior performance (95–99%+ accuracy) across a four-class dementia task. Key contributions include a comprehensive benchmarking of diverse architectures on the OASIS-based dataset, and the demonstration that a hybrid ensemble can dramatically reduce misclassification and improve ROC AUC, F1, and overall reliability. The findings suggest that hybrid CNN–Transformer models can provide clinically meaningful, deployable tools for AD staging with strong generalization across dementia categories.

Abstract

Early and accurate classification of Alzheimers disease (AD) from brain MRI scans is essential for timely clinical intervention and improved patient outcomes. This study presents a comprehensive comparative analysis of five CNN architectures (EfficientNetB0, ResNet50, DenseNet201, MobileNetV3, VGG16), five Transformer-based models (ViT, ConvTransformer, PatchTransformer, MLP-Mixer, SimpleTransformer), and a proposed hybrid model named Evan_V2. All models were evaluated on a four-class AD classification task comprising Mild Dementia, Moderate Dementia, Non-Demented, and Very Mild Dementia categories. Experimental findings show that CNN architectures consistently achieved strong performance, with ResNet50 attaining 98.83% accuracy. Transformer models demonstrated competitive generalization capabilities, with ViT achieving the highest accuracy among them at 95.38%. However, individual Transformer variants exhibited greater class-specific instability. The proposed Evan_V2 hybrid model, which integrates outputs from ten CNN and Transformer architectures through feature-level fusion, achieved the best overall performance with 99.99% accuracy, 0.9989 F1-score, and 0.9968 ROC AUC. Confusion matrix analysis further confirmed that Evan_V2 substantially reduced misclassification across all dementia stages, outperforming every standalone model. These findings highlight the potential of hybrid ensemble strategies in producing highly reliable and clinically meaningful diagnostic tools for Alzheimers disease classification.

Figures (8)

• Figure 1: System Design
• Figure 2: Data Sample
• Figure 3: t-SNE Visualization of Alzheimer MRI Images
• Figure 4: Architecture pipeline of the proposed Evan_V2 Hybrid Model integrating CNN and Transformer components.
• Figure 5: Confusion Matrix of CNN based models