US \& MRI Image Fusion Based on Markerless Skin Registration

TL;DR

The system eliminates the need for external physical markers and complex training, making image fusion feasible for physicians with different experience levels, and validates the system's accuracy, computational efficiency, noise robustness, and operator independence.

Abstract

This paper presents an innovative automatic fusion imaging system that combines 3D CT/MR images with real-time ultrasound (US) acquisition. The system eliminates the need for external physical markers and complex training, making image fusion feasible for physicians with different experience levels. The integrated system involves a portable 3D camera for patient-specific surface acquisition, an electromagnetic tracking system, and US components. The fusion algorithm comprises two main parts: skin segmentation and rigid co-registration, both integrated into the US machine. The co-registration software aligns the surface extracted from CT/MR images with patient-specific coordinates, facilitating rapid and effective fusion. Experimental testing in different settings validates the system's accuracy, computational efficiency, noise robustness, and operator independence. The co-registration error remains under the acceptable range of~$1$ cm.

Figures (12)

• Figure 1: (a) Hardware and software components of the system and their mutual interaction for the image fusion. (b) US system integrated into a testing setup that mimics the clinical environment.
• Figure 2: Description of the segmentation method on a slice. The toy image is on the left of each step, and the mock-up grid is on the right. The red pixels are under evaluation, and the orange pixels are inserted in the "visited pixel" list by the current evaluation step. The lighter orange pixels are inserted in the list by previous steps. (a) The algorithm's initialisation (every pixel in the mock-up gridhas a value of 2), evaluation of the first pixel, identification of the neighbourhood and consequent assignment of background value on the mock-up grid. (b) The second step of the algorithm has the same consideration as the previous one. (c) Identification of a pixel above the threshold: the value of the mock-up pixel changes to 1, and the neighbourhood of the pixel is not inserted in the list. (d) Final segmentation.
• Figure 3: Co-registration steps. (a) The selected anterior portion of the segmented surface. (b) PCA alignment and translation on the reference points. (c) Surface sub-regions for the first ICP run. (d) Surface sub-regions tuning for the second ICP step (only $80\%$ of the surface considered in the first ICP run is considered in the second refinement run).
• Figure 4: (a) Co-registration pipeline result. Error distribution (b) on the segmented surface and (c) on the camera mesh. The unit measure of the colourmap is mm.
• Figure 5: Robustness to image subsampling of the skin segmentation method. Skin surface extracted from the segmentation of the input image at (a) high and (b) low resolution. (c) Distance distribution between the two surfaces; the colourmap scale goes from $0$ mm (blue) to $5$ mm (red).