Efficient Complex-Valued Vision Transformers for MRI Classification Directly from k-Space

TL;DR

This work tackles the challenge of MRI classification by operating directly on raw, undersampled k-Space data, thereby preserving phase information often discarded in image-domain analyses. It introduces kViT, a fully complex-valued Vision Transformer that uses radial k-Space patching and complex-valued embeddings to model the global, non-local structure of frequency data. The approach achieves competitive classification performance against image-domain baselines while drastically reducing VRAM requirements (up to 68× in some settings) and demonstrating robustness to high undersampling across multiple datasets. These results suggest a practical, resource-efficient pathway for direct-from-scanner AI in MRI, with potential benefits for faster protocols and broader accessibility in clinical settings.

Abstract

Deep learning applications in Magnetic Resonance Imaging (MRI) predominantly operate on reconstructed magnitude images, a process that discards phase information and requires computationally expensive transforms. Standard neural network architectures rely on local operations (convolutions or grid-patches) that are ill-suited for the global, non-local nature of raw frequency-domain (k-Space) data. In this work, we propose a novel complex-valued Vision Transformer (kViT) designed to perform classification directly on k-Space data. To bridge the geometric disconnect between current architectures and MRI physics, we introduce a radial k-Space patching strategy that respects the spectral energy distribution of the frequency-domain. Extensive experiments on the fastMRI and in-house datasets demonstrate that our approach achieves classification performance competitive with state-of-the-art image-domain baselines (ResNet, EfficientNet, ViT). Crucially, kViT exhibits superior robustness to high acceleration factors and offers a paradigm shift in computational efficiency, reducing VRAM consumption during training by up to 68$\times$ compared to standard methods. This establishes a pathway for resource-efficient, direct-from-scanner AI analysis.

Figures (9)

• Figure 1: Illustration of the proposed radial k-Space patches and their application within the complex-valued transformer architecture.
• Figure 2: Classification results on the fastMRI prostate dataset. Note the consistent performance at higher acceleration factors. The shown results are the mean and standard deviation over five folds tested on a hold-out test set.
• Figure 3: The number of parameters and VRAM consumption during training for different model configurations on the fastMRI prostate and knee dataset at a batch size of 64. The proposed method achieves competitive performance with significantly reduced VRAM usage.
• Figure 4: Classification results on the fastMRI knee dataset. The proposed model reaches comparable performance to the SOTA at reduced VRAM consumption, but showing the same decline in performance at higher acceleration factors as the baselines. The shown results are the mean and standard deviation over five folds tested on a hold-out test set.
• Figure 5: Attention Maps from the last Transformer layer of the proposed method on the (left) fastMRI prostate dataset and (right) fastMRI knee dataset and the corresponding input data transformed into the image space magnitude. The attention maps are visualized logarithmically for better visibility. The model focuses primarily on the center region of k-Space, while also attending to high-frequency components in the outer regions.