Diffusion-based Generative Image Outpainting for Recovery of FOV-Truncated CT Images

TL;DR

This work trains a diffusion model and applies it to truncated CT slices generated by simulating a small FOV and reliably recovers the truncated anatomy and outperforms the previous state-of-the-art despite being trained on 87% less data.

Abstract

Field-of-view (FOV) recovery of truncated chest CT scans is crucial for accurate body composition analysis, which involves quantifying skeletal muscle and subcutaneous adipose tissue (SAT) on CT slices. This, in turn, enables disease prognostication. Here, we present a method for recovering truncated CT slices using generative image outpainting. We train a diffusion model and apply it to truncated CT slices generated by simulating a small FOV. Our model reliably recovers the truncated anatomy and outperforms the previous state-of-the-art despite being trained on 87% less data.

Figures (4)

• Figure 1: An overview of CT-Palette. The body bounding box detector estimates the bounding box representing the complete/untruncated body, and the image outpainting model recovers the tissues of the truncated CT slice. The white region in the mask indicates the region to be outpainted. CT-Palette generates different slices at each run by drawing samples from the distribution it has learned during training. Using a body composition segmentation model, we extract the muscle and SAT areas from each slice. The final output is the slice with muscle and SAT areas closest to the median.
• Figure 2: Synthetic data generation process: 1) To train the image outpainting model, we generate truncated slices and their corresponding small masks (explained above); 2) To train the body bounding box detector, we generate truncated slices cropped at the display FOV (DFOV) and bounding boxes of the untruncated bodies.
• Figure 3: Visual comparison of outpainted slices from different vertebral levels by various models, given a truncated CT slice. (FT): Fine-tuned. (MI): Multiple inference. CT-Palette recovers the truncated slice most realistically. Particularly in the T5 example, CT-Palette successfully restores the shoulder blade, as indicated by the red arrow.
• Figure 4: Box plots of the differences in muscle and SAT areas between ground-truth untruncated and outpainted slices by various models. The topmost box plots illustrate the difference in area between the untruncated and truncated slices for reference. Positive values indicate underestimation, negative values indicate overestimation, and 0 indicates no difference. CT-Palette has a median close to zero and very narrow IQR, suggesting little to no deviation from ground-truth in muscle and SAT areas.