Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

New multimodal similarity measure for image registration via modeling local functional dependence with linear combination of learned basis functions

Joel Honkamaa, Pekka Marttinen

TL;DR

This work addresses robust multimodal deformable image registration by introducing a local similarity measure based on residuals from a locally fitted function. The main idea is to model the local intensity relationship as a linear combination of learned basis functions $f_{4theta_r,omega}$, with coefficients $\widehat{\theta}_r$ solvable in closed form and the basis functions learned jointly with the deformation; the overall loss is computed efficiently via convolutions, enabling GPU acceleration. The authors provide a practical multi-resolution pipeline, augment the inputs with derivative channels, and demonstrate superior or competitive performance on three datasets compared to strong baselines, with code and data availability. This approach offers a scalable, effective multiscale method for cross-modality registration, potentially improving clinical workflows that rely on integrating information from different imaging modalities.

Abstract

The deformable registration of images of different modalities, essential in many medical imaging applications, remains challenging. The main challenge is developing a robust measure for image overlap despite the compared images capturing different aspects of the underlying tissue. Here, we explore similarity metrics based on functional dependence between intensity values of registered images. Although functional dependence is too restrictive on the global scale, earlier work has shown competitive performance in deformable registration when such measures are applied over small enough contexts. We confirm this finding and further develop the idea by modeling local functional dependence via the linear basis function model with the basis functions learned jointly with the deformation. The measure can be implemented via convolutions, making it efficient to compute on GPUs. We release the method as an easy-to-use tool and show good performance on three datasets compared to well-established baseline and earlier functional dependence-based methods.

New multimodal similarity measure for image registration via modeling local functional dependence with linear combination of learned basis functions

TL;DR

This work addresses robust multimodal deformable image registration by introducing a local similarity measure based on residuals from a locally fitted function. The main idea is to model the local intensity relationship as a linear combination of learned basis functions , with coefficients solvable in closed form and the basis functions learned jointly with the deformation; the overall loss is computed efficiently via convolutions, enabling GPU acceleration. The authors provide a practical multi-resolution pipeline, augment the inputs with derivative channels, and demonstrate superior or competitive performance on three datasets compared to strong baselines, with code and data availability. This approach offers a scalable, effective multiscale method for cross-modality registration, potentially improving clinical workflows that rely on integrating information from different imaging modalities.

Abstract

The deformable registration of images of different modalities, essential in many medical imaging applications, remains challenging. The main challenge is developing a robust measure for image overlap despite the compared images capturing different aspects of the underlying tissue. Here, we explore similarity metrics based on functional dependence between intensity values of registered images. Although functional dependence is too restrictive on the global scale, earlier work has shown competitive performance in deformable registration when such measures are applied over small enough contexts. We confirm this finding and further develop the idea by modeling local functional dependence via the linear basis function model with the basis functions learned jointly with the deformation. The measure can be implemented via convolutions, making it efficient to compute on GPUs. We release the method as an easy-to-use tool and show good performance on three datasets compared to well-established baseline and earlier functional dependence-based methods.

Paper Structure

This paper contains 10 sections, 9 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Notation
  3. Related work
  4. Methods
  5. Further details
  6. Experiments
  7. Hyperparameter optimization
  8. Comparison to earlier methods
  9. Ablation study
  10. Discussion

Figures (1)

  • Figure 1: We measure multimodal similarity via residuals of locally fitted functions (over sliding window). Each point on the right describes intensity value pair for voxels at identical locations in the patches (only one slice of the 3D volumes is shown). The points are weighted by the distance from the patch center. Blue curve: Learning basis (globally) allows reasonably good fit with very few basis functions (number of terms $J=5$). Red curve: Polynomial functions (number of terms $J=6$) struggle to fit the high frequencies. © Copyright CERMEP – Imagerie du vivant, www.cermep.fr and Hospices Civils de Lyon. All rights reserved.