Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Investigating the Feasibility of Patch-based Inference for Generalized Diffusion Priors in Inverse Problems for Medical Images

Saikat Roy, Mahmoud Mostapha, Radu Miron, Matt Holbrook, Mariappan Nadar

TL;DR

This work evaluates patch-based diffusion priors for MRI inverse problems, proposing a single generalized prior trained on a large, diverse MRI dataset to support restoration and super-resolution across multiple anatomies. It introduces Shifted-Grid Inference to enable patch-based plug-and-play with various solvers and analyzes boundary-artifact mitigation, memory efficiency, and cross-domain applicability. The results show patch-trained priors can match full-image priors in performance while delivering meaningful memory savings, suggesting practicality for high-resolution MRI tasks. The findings inform guidelines for deploying patch-based diffusion priors in clinical imaging contexts and highlight directions for further reducing artifacts and extending to ultra-high-resolution data.

Abstract

Plug-and-play approaches to solving inverse problems such as restoration and super-resolution have recently benefited from Diffusion-based generative priors for natural as well as medical images. However, solutions often use the standard albeit computationally intensive route of training and inferring with the whole image on the diffusion prior. While patch-based approaches to evaluating diffusion priors in plug-and-play methods have received some interest, they remain an open area of study. In this work, we explore the feasibility of the usage of patches for training and inference of a diffusion prior on MRI images. We explore the minor adaptation necessary for artifact avoidance, the performance and the efficiency of memory usage of patch-based methods as well as the adaptability of whole image training to patch-based evaluation - evaluating across multiple plug-and-play methods, tasks and datasets.

Investigating the Feasibility of Patch-based Inference for Generalized Diffusion Priors in Inverse Problems for Medical Images

TL;DR

This work evaluates patch-based diffusion priors for MRI inverse problems, proposing a single generalized prior trained on a large, diverse MRI dataset to support restoration and super-resolution across multiple anatomies. It introduces Shifted-Grid Inference to enable patch-based plug-and-play with various solvers and analyzes boundary-artifact mitigation, memory efficiency, and cross-domain applicability. The results show patch-trained priors can match full-image priors in performance while delivering meaningful memory savings, suggesting practicality for high-resolution MRI tasks. The findings inform guidelines for deploying patch-based diffusion priors in clinical imaging contexts and highlight directions for further reducing artifacts and extending to ultra-high-resolution data.

Abstract

Plug-and-play approaches to solving inverse problems such as restoration and super-resolution have recently benefited from Diffusion-based generative priors for natural as well as medical images. However, solutions often use the standard albeit computationally intensive route of training and inferring with the whole image on the diffusion prior. While patch-based approaches to evaluating diffusion priors in plug-and-play methods have received some interest, they remain an open area of study. In this work, we explore the feasibility of the usage of patches for training and inference of a diffusion prior on MRI images. We explore the minor adaptation necessary for artifact avoidance, the performance and the efficiency of memory usage of patch-based methods as well as the adaptability of whole image training to patch-based evaluation - evaluating across multiple plug-and-play methods, tasks and datasets.
Paper Structure (12 sections, 1 equation, 3 figures, 1 table)

This paper contains 12 sections, 1 equation, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Training of a Single Generalized Diffusion Prior
  4. Shifted-Grid Inference
  5. Avoidance of Foreground-to-Background Boundary Transition Artifacts
  6. Experimental Setup
  7. Results and Discussion
  8. Patch-wise trained models offer comparable performance to whole image models
  9. Generalized Patch-based inference
  10. Patch-wise plug-and-play prefers patchwise training
  11. Patch-wise inference offers efficient memory usage
  12. Conclusion

Figures (3)

  • Figure 1: Foreground-to-Background Boundary Transistion Artifacts during patchwise inference. These artifacts are seen on the left edge of the image in the case of zero-padding (left) while being absent for reflection padding (right).
  • Figure 2: Results on Denoising (Top row) and Super-Resolution (Bottom row) tasks using DiffPIR. We demonstrate perceptually comparable performance of our diverse prior in both patch-based (DiffPIR$_\text{full}$) and patch-based DiffPIR$_\text{patch}$ training.
  • Figure 3: Patch-based inference leads to noticeable memory reduction during inference. This is as much as 25% when moving from $320 \times 320$ to $128 \times 128$ patches in DiffPIR.