From Pixel to Cancer: Cellular Automata in Computed Tomography

TL;DR

Pixel2Cancer reframes tumor synthesis as a three-rule cellular automata process applied to CT images to generate multi-stage tumors that can be mapped into real scans. It introduces a four-level organ quantification plus invasion dynamics, enabling cross-organ tumor generation without manual annotations. The study demonstrates clinically realistic synthetic tumors via a Visual Turing Test and shows segmentation performance surpassing baselines on liver, pancreas, and kidney across a large, multi-institution dataset. The approach promises improved data augmentation and early cancer detection by enabling controlled generation of tumors and their trajectories in longitudinal data.

Abstract

AI for cancer detection encounters the bottleneck of data scarcity, annotation difficulty, and low prevalence of early tumors. Tumor synthesis seeks to create artificial tumors in medical images, which can greatly diversify the data and annotations for AI training. However, current tumor synthesis approaches are not applicable across different organs due to their need for specific expertise and design. This paper establishes a set of generic rules to simulate tumor development. Each cell (pixel) is initially assigned a state between zero and ten to represent the tumor population, and a tumor can be developed based on three rules to describe the process of growth, invasion, and death. We apply these three generic rules to simulate tumor development--from pixel to cancer--using cellular automata. We then integrate the tumor state into the original computed tomography (CT) images to generate synthetic tumors across different organs. This tumor synthesis approach allows for sampling tumors at multiple stages and analyzing tumor-organ interaction. Clinically, a reader study involving three expert radiologists reveals that the synthetic tumors and their developing trajectories are convincingly realistic. Technically, we analyze and simulate tumor development at various stages using 9,262 raw, unlabeled CT images sourced from 68 hospitals worldwide. The performance in segmenting tumors in the liver, pancreas, and kidneys exceeds prevailing literature benchmarks, underlining the immense potential of tumor synthesis, especially for earlier cancer detection. The code and models are available at https://github.com/MrGiovanni/Pixel2Cancer

Figures (6)

• Figure 1: Initially, we quantify the organ from CT intensity to create the quantified organ and select a pixel as the starting point. Subsequently, we apply three generic rules to simulate tumor development based on the quantified organ, including growth, interaction, and death (§\ref{['sec:three_rules']}). Then, using these generic rules, Pixel2Cancer simulates tumor development and records simulation results using a tumor population map (§\ref{['sec:tumor_organ_interaction']}). Finally, we generate tumors in CT scans through the mapping function according to the tumor population map and CT intensity (§\ref{['sec:map_back_ct']}).
• Figure 2: R1. Growth: Tumor cells can proliferate themselves ($\text{self-state} +1$) with probability. R2. Invasion: Tumor cells can invade neighboring cells ($\text{neighbor-state} +1$). In this process, we simulate interactions among tumors, organ tissues, vessels, and boundaries. At the bottom line, we present cases where tumors are compressed by organ boundaries and vessels. R3. Death: Tumor cells surrounded by a full population of neighboring cells ($\text{state}=10$) will undergo cell death ($\text{self-state}\leftarrow -1$).
• Figure 3: (a) Different condition: We evaluate the influence of different tumor sizes, intensities, and roundness on liver tumors to assess the robustness of Pixel2Cancer. (b) Early tumor detection: We report the Sensitivity of U-Net tumor detection among real tumors, Hu et al. hu2023label, and Pixel2Cancer in three organs. (c) Effectiveness of rules: We evaluated the performance of each generic rule on liver tumors, demonstrating the effectiveness of our rule design in tumor simulation.
• Figure 4: Examples of early tumor detection. Qualitative visualizations of segmentation models for liver, pancreas, and kidney tumor detection.
• Figure 5: Examples of Visual Turning Test. Tubular results are presented in \ref{['tab:reader_studies']}. Our Pixel2Cancer can be used to augment available healthy CT volumes.