Imaging foundation model for universal enhancement of non-ideal measurement CT

TL;DR

This work introduces TAMP, a universal imaging foundation model for non-ideal measurement CT enhancement, addressing generalizability and data efficiency across LDCT, SVCT, and LACT settings. It combines physics-driven pre-training on a massive SimNICT dataset ($3{,}633{,}374$ images from $9{,}638$ ICT volumes; $10.8$ million NICT-ICT pairs) with a multi-scale transformer architecture (MITNet) and dual-domain enhancement learning (DDEL), enabling robust, universal enhancement. Efficient task adaptation is achieved via LoRA, allowing fine-tuning from as little as $5$ slices to specialized clinical scenarios, with rapid convergence. Real-world validations and radiologist studies show improved image quality, edge preservation, and clinical acceptability, highlighting TAMP’s potential to broaden NICT applications and reduce development costs for specialized enhancement models.

Abstract

Non-ideal measurement computed tomography (NICT) employs suboptimal imaging protocols to expand CT applications. However, the resulting trade-offs degrade image quality, limiting clinical acceptability. Although deep learning methods have been used to enhance NICT images, their reliance on large training datasets and limited generalizability across diverse settings hinder practical use. We propose the multi-scale integrated Transformer AMPlifier (TAMP), the first imaging foundation model for universal NICT enhancement. Pre-trained on 10.8 million physics-driven simulated NICT images, TAMP generalizes effectively across various NICT settings, defect degrees, and body regions. Moreover, a parameter-efficient fine-tuning strategy enables TAMP to adapt to specific clinical scenarios using only few slices. Extensive experiments, including radiologists and real-world validations, demonstrate that TAMP consistently improves image quality and clinical acceptability, underscoring its significant potential to advance CT imaging and broaden NICT applications in clinical practice.

Figures (26)

• Figure 1: Our TAMP is a universal non-ideal measurement computed tomography (NICT) enhancement foundation model that is able to enhance NICT images with various body regions and non-ideal settings and improve the efficiency of developing specialized NICT enhancement models. a) NICT expands the scope of CT applications with the advantages of radiation dose reduction, scanning acceleration, and adaptation of restricted scanning posture. However, the image quality of NICT is reduced, limiting its clinical effectiveness. b) Specialized NICT enhancement models that focus on specific body regions or non-ideal settings, are limited by the application scopes and construction costs. c) Our TAMP can directly enhance NICT images and adapt to specialized NICT enhancement tasks with low data and computational costs, improving imaging applications' development efficiency and effectiveness.
• Figure 2: Our TAMP enhances diverse NICT settings with varying defect degrees and body regions. a-b) Our TAMP leverages the largest training dataset (SimNICT), spanning diverse NICT settings and body regions. c-d) Our TAMP achieves universal NICT enhancement surpassing specialized models across three body regions, three NICT settings, and three defect degrees. e) Our TAMP significantly improves NICT image quality, preserving fine structural details and enhancing their clinical utility.
• Figure 3: TAMP achieves efficient adaptation to specific NICT settings with low data and computational costs. a) Only adapted with 5 slices, our TAMP has significantly outperformed the compared methods. b) Our TAMP can significantly reduce the training data requirement. It only needs 5 slices to achieve comparable or even better performance than the comparison methods with 20 volumes. c) Adapted with 5 slices, TAMP achieves convergence within a small number of epochs, thereby effectively reducing training iterations.
• Figure 4: Real-world validation demonstrates our great application potential in real-world NICT images. a) Our TAMP can directly enhance real-world NICT images and TAMP-S is adapted to specific NICT settings, achieving further improvement. b) Our TAMP has significant visual enhancement demonstrating its great clinical application potential in real-world NICT images.
• Figure 5: The radiologist validation illustrates our superior clinical acceptance. a) The process of our radiologist validation. b-d) We designed three metrics to quantify this acceptance, i.e., the probability of being better than NICT, subjective quality ranking, and probability of clinical acceptance.