Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A self-attention model for robust rigid slice-to-volume registration of functional MRI

Samah Khawaled, Simon K. Warfield, Moti Freiman

TL;DR

This work tackles intra-volume head-motion in fMRI by formulating slice-to-volume registration (SVR) as aligning a stack of 2D slices with a 3D reference under rigid motion. It introduces a self-attention SVR (SA-SVR) that uses dual-branch encoders and a transformer to assign per-slice scores, weighing inputs by uncertainty and improving robustness. Trained on synthetic motions derived from the Healthy Brain Network dataset, SA-SVR achieves a grid-distance of about $0.93\text{ mm}$ and faster runtimes ($\sim$0.096 s) compared to conventional iterative methods, while outperforming a state-of-the-art DL-based baseline in several metrics. The approach enables near-real-time head-motion monitoring during acquisition and can be extended to unsupervised learning for broader clinical deployment.

Abstract

Functional Magnetic Resonance Imaging (fMRI) is vital in neuroscience, enabling investigations into brain disorders, treatment monitoring, and brain function mapping. However, head motion during fMRI scans, occurring between shots of slice acquisition, can result in distortion, biased analyses, and increased costs due to the need for scan repetitions. Therefore, retrospective slice-level motion correction through slice-to-volume registration (SVR) is crucial. Previous studies have utilized deep learning (DL) based models to address the SVR task; however, they overlooked the uncertainty stemming from the input stack of slices and did not assign weighting or scoring to each slice. In this work, we introduce an end-to-end SVR model for aligning 2D fMRI slices with a 3D reference volume, incorporating a self-attention mechanism to enhance robustness against input data variations and uncertainties. It utilizes independent slice and volume encoders and a self-attention module to assign pixel-wise scores for each slice. We conducted evaluation experiments on 200 images involving synthetic rigid motion generated from 27 subjects belonging to the test set, from the publicly available Healthy Brain Network (HBN) dataset. Our experimental results demonstrate that our model achieves competitive performance in terms of alignment accuracy compared to state-of-the-art deep learning-based methods (Euclidean distance of $0.93$ [mm] vs. $1.86$ [mm]). Furthermore, our approach exhibits significantly faster registration speed compared to conventional iterative methods ($0.096$ sec. vs. $1.17$ sec.). Our end-to-end SVR model facilitates real-time head motion tracking during fMRI acquisition, ensuring reliability and robustness against uncertainties in inputs. source code, which includes the training and evaluations, will be available soon.

A self-attention model for robust rigid slice-to-volume registration of functional MRI

TL;DR

This work tackles intra-volume head-motion in fMRI by formulating slice-to-volume registration (SVR) as aligning a stack of 2D slices with a 3D reference under rigid motion. It introduces a self-attention SVR (SA-SVR) that uses dual-branch encoders and a transformer to assign per-slice scores, weighing inputs by uncertainty and improving robustness. Trained on synthetic motions derived from the Healthy Brain Network dataset, SA-SVR achieves a grid-distance of about and faster runtimes (0.096 s) compared to conventional iterative methods, while outperforming a state-of-the-art DL-based baseline in several metrics. The approach enables near-real-time head-motion monitoring during acquisition and can be extended to unsupervised learning for broader clinical deployment.

Abstract

Functional Magnetic Resonance Imaging (fMRI) is vital in neuroscience, enabling investigations into brain disorders, treatment monitoring, and brain function mapping. However, head motion during fMRI scans, occurring between shots of slice acquisition, can result in distortion, biased analyses, and increased costs due to the need for scan repetitions. Therefore, retrospective slice-level motion correction through slice-to-volume registration (SVR) is crucial. Previous studies have utilized deep learning (DL) based models to address the SVR task; however, they overlooked the uncertainty stemming from the input stack of slices and did not assign weighting or scoring to each slice. In this work, we introduce an end-to-end SVR model for aligning 2D fMRI slices with a 3D reference volume, incorporating a self-attention mechanism to enhance robustness against input data variations and uncertainties. It utilizes independent slice and volume encoders and a self-attention module to assign pixel-wise scores for each slice. We conducted evaluation experiments on 200 images involving synthetic rigid motion generated from 27 subjects belonging to the test set, from the publicly available Healthy Brain Network (HBN) dataset. Our experimental results demonstrate that our model achieves competitive performance in terms of alignment accuracy compared to state-of-the-art deep learning-based methods (Euclidean distance of [mm] vs. [mm]). Furthermore, our approach exhibits significantly faster registration speed compared to conventional iterative methods ( sec. vs. sec.). Our end-to-end SVR model facilitates real-time head motion tracking during fMRI acquisition, ensuring reliability and robustness against uncertainties in inputs. source code, which includes the training and evaluations, will be available soon.
Paper Structure (16 sections, 3 equations, 5 figures, 4 tables)

This paper contains 16 sections, 3 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Methods
  4. The Network Architecture
  5. The Self-Attention Model
  6. The Slices and Volume Encoders
  7. The Regression Model
  8. Representations of Transformations and Slices Generation
  9. Loss Function
  10. Experimental Methodology
  11. Database and Preprocessing
  12. Implementation Details
  13. fMRI Motion Simulation Study
  14. Evaluation Methodology
  15. Results and Discussion
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Block diagram depicting our SA-SVR system. A self-attention transformer predicts a score for each slice in the stack, followed by the multiplication of each slice by its respective score. Features extracted from both the dual-branch slices and volume encoders are concatenated and employed for the prediction of rigid parameters.
  • Figure 2: A detailed Architecture of the self-attention transformer. The first linear layer embeds the sequence input into a hidden dimension space of $D=256$
  • Figure 3: Illustration of the SVR problem. Two examples of free-motion volumes and volumes with synthetic motion. From left to right: Coronal \ref{['fig1:a']},\ref{['fig1:b']} and Sagittal \ref{['fig1:c']},\ref{['fig1:d']} views of the free-motion volumes, the generated volumes after applying the rigid transformations and sampling the slices, and the pixel-wise MSE between them, respectively. The artifacts due to the slice-level motion are visible in both Sagittal and Coronal views, however, they are not dipicted in the axial (imaging) axis of the image \ref{['fig1:e']},\ref{['fig1:f']}.
  • Figure 4: fMRI motion-correction results. Sampled intensity values from various voxels across the registered volume (marked as AR - after registration), the motion-free volume ($V_t$), and the volume with simulated motion (denoted as BR - before registration), collected over time points representing the course of the fMRI series. The voxels used in the analysis for the upper and bottom parts were sampled from the left and right red boxes in Fig. \ref{['fig:pixelrois']}, respectively.
  • Figure 5: ROIs of the selected pixels. From the highlighted red regions, we sampled voxels used for the fMRI motion correction experiment. The upper and bottom figures in Fig. \ref{['fig:pixelfmri']} show results on voxels sampled from the left and right red boxes, respectively.