Parameter Efficient Fine Tuning for Multi-scanner PET to PET Reconstruction

TL;DR

This study tackles the challenge of reducing PET scan time while preserving image quality across multiple scanners. It introduces PETITE, a parameter-efficient fine-tuning framework for multi-scanner PET-to-PET reconstruction that leverages Mix-PEFT on ViT-based encoder–decoder models, achieving near Full-FT performance with less than 1% of parameters. The method demonstrates substantial improvements over individual PEFTs in quantitative metrics (PSNR, SSIM, NRMSE) and qualitative reconstructions, with optimal encoder/decoder configurations identified for both 3D CVT-GAN and UNETR architectures. Practically, PETITE enables efficient cross-scanner adaptation and shorter scans, reducing resource demands while maintaining clinical-quality reconstructions; it lays the groundwork for broader adoption of parameter-efficient techniques in medical imaging reconstruction.

Abstract

Reducing scan time in Positron Emission Tomography (PET) imaging while maintaining high-quality images is crucial for minimizing patient discomfort and radiation exposure. Due to the limited size of datasets and distribution discrepancy across scanners in medical imaging, fine-tuning in a parameter-efficient and effective manner is on the rise. Motivated by the potential of Parameter-Efficient Fine-Tuning (PEFT), we aim to address these issues by effectively leveraging PEFT to improve limited data and GPU resource issues in multi-scanner setups. In this paper, we introduce PETITE, Parameter-Efficient Fine-Tuning for MultI-scanner PET to PET REconstruction that uses fewer than 1% of the parameters. To the best of our knowledge, this study is the first to systematically explore the efficacy of diverse PEFT techniques in medical imaging reconstruction tasks via prevalent encoder-decoder-type deep models. This investigation, in particular, brings intriguing insights into PETITE as we show further improvements by treating encoder and decoder separately and mixing different PEFT methods, namely, Mix-PEFT. Using multi-scanner PET datasets comprised of five different scanners, we extensively test the cross-scanner PET scan time reduction performances (i.e., a model pre-trained on one scanner is fine-tuned on a different scanner) of 21 feasible Mix-PEFT combinations to derive optimal PETITE. We show that training with less than 1% parameters using PETITE performs on par with full fine-tuning (i.e., 100% parameter)

Figures (4)

• Figure 1: Illustration of multi-scanner PET scan time reduction with PEFT
• Figure 2: The pipeline of the encoder-decoder structure of each ViT-based model. (a) 3D CVT-GAN cvtgan features a generator with a ViT-based encoder and decoder. Only the first three layers of the encoder and the first two layers of the decoder are trained. (b) UNETR unetr consists of a ViT-based encoder and a CNN-based decoder.
• Figure 3: Illustrations of the modified structures of PEFT methods.
• Figure 4: Scan time reduction examples using 3D CVT-GAN cvtgan and UNETR unetr.First row: PET scans. Second row: error maps comparing the reconstructed PET scans to the ground-truth (GT).