Neural Graphics Primitives-based Deformable Image Registration for On-the-fly Motion Extraction

TL;DR

This work presents NGPDIR, a per-case, self-supervised deformable image registration approach that combines INR-based DIR with Neural Graphics Primitives (NGP) to optimize the displacement vector field (DVF) for 4D-CT lung motion. By encoding DVFs with multi-resolution hash tables and a shallow network, NGPDIR achieves resolution-independent, fast DVF estimation, addressing sliding boundary challenges without requiring large pre-training. On the DIR-lab 4D-CT dataset, NGPDIR attains a target registration error of $1.15\pm1.15$ mm in $1.77$ seconds, representing a state-of-the-art speed-accuracy trade-off and opening avenues for real-time motion extraction in radiotherapy. The approach combines per-case optimization with strong potential for real-time adaptive planning, motion tracking, and dose optimization, while highlighting opportunities for meta-learning to generalize priors across anatomical regions.

Abstract

Intra-fraction motion in radiotherapy is commonly modeled using deformable image registration (DIR). However, existing methods often struggle to balance speed and accuracy, limiting their applicability in clinical scenarios. This study introduces a novel approach that harnesses Neural Graphics Primitives (NGP) to optimize the displacement vector field (DVF). Our method leverages learned primitives, processed as splats, and interpolates within space using a shallow neural network. Uniquely, it enables self-supervised optimization at an ultra-fast speed, negating the need for pre-training on extensive datasets and allowing seamless adaptation to new cases. We validated this approach on the 4D-CT lung dataset DIR-lab, achieving a target registration error (TRE) of 1.15\pm1.15 mm within a remarkable time of 1.77 seconds. Notably, our method also addresses the sliding boundary problem, a common challenge in conventional DIR methods.

Figures (5)

• Figure 1: Landmark error, averaged over all cases, for different DIR methods. Conventional and DL-based methods are denoted in green and red, respectively, with reported values from the literature. INR/NGP based (blue) have been implemented by us and exclusively trained on the lung region for comparison.
• Figure 2: Schematic of the NGPDIR framework. A random point (x, y, z) is represented as the interpolations of multi-level primitives, then mapped to the displacement vector by a small network. The primitives and the network are optimized with respect to each image.
• Figure 3: MAE and Dice scores for the different DIR methods averaged over all cases. Orange lines in sub-figures (b-c) represent the absence of registration as a baseline. Solid lines indicate results for the vertebra, while dashed lines correspond to ribs.
• Figure 4: Visualization of landmark movement for case 8. Ground-truth motions are denoted by green arrows, predictions by blue, and errors by red. The magnitude of errors corresponds to the length of the red arrows. These projections are in the coronal plane for clarity.
• Figure 5: Example DVFs and error maps for case 8. ccIDIR, IDIR, and NGPDIR are compared with similar training times (around 3.65s). DVFs are color-coded for directionality, with the intensity reflecting the magnitude. Green arrows point to ribs (sliding boundaries).