Neural Image Unfolding: Flattening Sparse Anatomical Structures using Neural Fields

TL;DR

This work deploys a neural field to fit the transformation of the anatomy of interest to a 2D overview image and proposes distortion regularization strategies and combines geometric with intensity-based loss formulations to also display non-annotated and auxiliary targets.

Abstract

Tomographic imaging reveals internal structures of 3D objects and is crucial for medical diagnoses. Visualizing the morphology and appearance of non-planar sparse anatomical structures that extend over multiple 2D slices in tomographic volumes is inherently difficult but valuable for decision-making and reporting. Hence, various organ-specific unfolding techniques exist to map their densely sampled 3D surfaces to a distortion-minimized 2D representation. However, there is no versatile framework to flatten complex sparse structures including vascular, duct or bone systems. We deploy a neural field to fit the transformation of the anatomy of interest to a 2D overview image. We further propose distortion regularization strategies and combine geometric with intensity-based loss formulations to also display non-annotated and auxiliary targets. In addition to improved versatility, our unfolding technique outperforms mesh-based baselines for sparse structures w.r.t. peak distortion and our regularization scheme yields smoother transformations compared to Jacobian formulations from neural field-based image registration.

Figures (11)

• Figure 1: Unfolding sparse cerebral vessels using a neural field $f$: Fitting a 2-dimensional manifold in the tomographic image $I$ from a sparse set of automatically extracted 3D target centerline points $\mathbf{T}$ (red). Three unfolding examples on the right with depicted centerlines.
• Figure 2: Neural unfolding pipeline for the fitting process using randomly sampled points (top) and for inference using a grid input leading to a structured image after read-out (bottom).
• Figure 3: Importance sampling (left) and map overlay (right) from an importance definition in volumetric space which is inversely proportional to the Euclidean distance to the target (green).
• Figure 4: Neural unfolding results (first row) compared to two baselines, CeVasMap with spline predeformation (center) cevasmap and with planar initialization equivalent to an ARAP deformation (bottom row) Arap. Unfolded images are presented together with their read-out mesh showing two different views. The target points shown with the mesh are depicted as image overlays in Fig. \ref{['fig:teaser']} for improved clarity. Important aspects and artifacts are highlighted in yellow. All images have an isotropic image resolution of 0.5 mm.
• Figure 5: Logarithmic distortion distribution as boxplot (median line, std-dev. boxes, maximum whiskers) within 1 cm radius around unfolded target for each anatomy and method. Horizontal line of pixel spacing $\delta$, approx. indicating the beginning of severe distortions. Red dots represent max. distortion per case.