Multi-Objective Deep-Learning-based Biomechanical Deformable Image Registration with MOREA

TL;DR

This work integrates fast, deep-learning-based deformable image registration with a biomechanical, multi-objective optimization framework to address large deformations and content mismatch. By initializing MOREA’s FEM-like mesh transformation model with DL-MODIR predictions, the proposed DL-MOREA hybrid achieves high-quality registrations with reduced folding and improved bladder contour alignment, particularly for challenging cases, and faster convergence than MOREA alone. The study demonstrates the hybrid’s advantages on pelvic CT data with significant bladder changes, showing potential for clinical deployment where both realism and efficiency are critical. The results suggest that combining DL speed with biomechanical realism in a multi-objective setting can provide robust, clinically useful DIR solutions across different scenarios.

Abstract

When choosing a deformable image registration (DIR) approach for images with large deformations and content mismatch, the realism of found transformations often needs to be traded off against the required runtime. DIR approaches using deep learning (DL) techniques have shown remarkable promise in instantly predicting a transformation. However, on difficult registration problems, the realism of these transformations can fall short. DIR approaches using biomechanical, finite element modeling (FEM) techniques can find more realistic transformations, but tend to require much longer runtimes. This work proposes the first hybrid approach to combine them, with the aim of getting the best of both worlds. This hybrid approach, called DL-MOREA, combines a recently introduced multi-objective DL-based DIR approach which leverages the VoxelMorph framework, called DL-MODIR, with MOREA, an evolutionary algorithm-based, multi-objective DIR approach in which a FEM-like biomechanical mesh transformation model is used. In our proposed hybrid approach, the DL results are used to smartly initialize MOREA, with the aim of more efficiently optimizing its mesh transformation model. We empirically compare DL-MOREA against its components, DL-MODIR and MOREA, on CT scan pairs capturing large bladder filling differences of 15 cervical cancer patients. While MOREA requires a median runtime of 45 minutes, DL-MOREA can already find high-quality transformations after 5 minutes. Compared to the DL-MODIR transformations, the transformations found by DL-MOREA exhibit far less folding and improve or preserve the bladder contour distance error.

Figures (6)

• Figure 1: Illustration of the proposed hybrid DL-MOREA DIR approach and its main components.
• Figure 2: Reference source and target renders of the three selected patients. Shown are annotated sagittal slices.
• Figure 3: Scatter plot of the source and target volumes of the bladder of each tested patient. Patients are clustered into three clusters by their relative volume change between source and target, indicated in color in the plot. The red line indicates unchanged volume, while all data points below it reflect patients whose target bladder volume is smaller than their source bladder volume.
• Figure 4: Renders of the found transformations of all three approaches, showing transformed contours and inverse DVFs of automatically selected registrations. For MOREA and DL-MOREA, an early stop was simulated by storing the intermediate results of optimization after 5 min. Shown are annotated sagittal slices. Arrow color indicates deformation magnitude (see legend). The red rectangle indicates a region of interest in the analysis.
• Figure 5: Renders of the found transformations of all three approaches after convergence, showing transformed contours and inverse DVFs of automatically selected registrations. Shown are annotated sagittal slices. Arrow color indicates deformation magnitude (see legend).