Towards Saner Deep Image Registration

TL;DR

A novel regularization-based sanity-enforcer method is proposed that imposes two sanity checks on the deep model to reduce its inverse consistency errors and increase its discriminative power simultaneously.

Abstract

With recent advances in computing hardware and surges of deep-learning architectures, learning-based deep image registration methods have surpassed their traditional counterparts, in terms of metric performance and inference time. However, these methods focus on improving performance measurements such as Dice, resulting in less attention given to model behaviors that are equally desirable for registrations, especially for medical imaging. This paper investigates these behaviors for popular learning-based deep registrations under a sanity-checking microscope. We find that most existing registrations suffer from low inverse consistency and nondiscrimination of identical pairs due to overly optimized image similarities. To rectify these behaviors, we propose a novel regularization-based sanity-enforcer method that imposes two sanity checks on the deep model to reduce its inverse consistency errors and increase its discriminative power simultaneously. Moreover, we derive a set of theoretical guarantees for our sanity-checked image registration method, with experimental results supporting our theoretical findings and their effectiveness in increasing the sanity of models without sacrificing any performance. Our code and models are available at https://github.com/tuffr5/Saner-deep-registration.

Figures (12)

• Figure 1: FV-SDice-Dice comparisons of deep registrations on IXI Brain dataset. The vertical axis is FV (% of folded voxels), the horizontal axis is SDice (Self-Dice), and the circle size is Dice. Both sanity-checked models (VM-ESC and TMBS-ESC) achieve better diffeomorphism, competitive registration performance, and significantly improved self-sanity, compared to other models, including models with inverse consistency (ICNet zhang2018inverse, ICON Greer_2021_ICCV).
• Figure 2: Self-sanity error maps comparison. Left: with no self-sanity check, Right: with self-sanity check. We unify the error maps' scale bars for a fair comparison.
• Figure 3: Comparisons between strict inverse consistency trained results (Top) and cross-sanity checked results (Bottom). Our relaxed sanity-checked result maintains a similar level of inverse consistency as $\varphi^{m\rightarrow f}\circ\varphi^{f\rightarrow m}$ is close to id transformation (second column). We can also observe that ours produces a more regular map, compared to the folded map from the model trained with strict inverse consistency. Best view zoomed.
• Figure 4: Training a sanity-checked model. Ⓦ denotes spatial warping, e.g., warped $m$ is that we warp moving image $m$ using the transformation map calculated from $g^{m\rightarrow f}$.
• Figure 5: Mask evolution during training. Overall, as training proceeds, violators of the cross-sanity check are decreasing.

Theorems & Definitions (9)

• Definition 1: Ideal/Strict inverse consistency
• Theorem 1: Relaxed registration via cross-sanity check
• Theorem 2: Existence of the unique minimizer for our relaxed optimization
• Remark 2.1
• Theorem 3: Loyalty of the sanity-checked minimizer
• Lemma 4: Upper-bound of distance between optimal minimizer and sanity-checked minimizer
• Theorem 1: Relaxed ideal symmetric registration via cross-sanity check
• Theorem 2: Existence of the unique minimizer for our relaxed optimization
• Theorem 3: Loyalty of the sanity-checked minimizer