Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic-Guided Diffusion Model for Single-Step Image Super-Resolution

Zihang Liu, Zhenyu Zhang, Hao Tang

TL;DR

SAMSR addresses the bottleneck of single-step diffusion SR in semantically complex images by injecting semantic guidance from segmentation masks. It introduces the SAM-Noise Module to produce spatially adaptive noise and a pixel-wise forward process that modulates residual transfer and noise strength via a mask-derived weight $W(x, y)$. A semantic consistency loss aligns predicted and ground-truth semantic weights to accelerate training and improve region-aware reconstruction. Empirical results on real and synthetic datasets show improved perceptual quality and detail recovery in semantically rich regions, with faster convergence and competitive fidelity metrics.

Abstract

Diffusion-based image super-resolution (SR) methods have demonstrated remarkable performance. Recent advancements have introduced deterministic sampling processes that reduce inference from 15 iterative steps to a single step, thereby significantly improving the inference speed of existing diffusion models. However, their efficiency remains limited when handling complex semantic regions due to the single-step inference. To address this limitation, we propose SAMSR, a semantic-guided diffusion framework that incorporates semantic segmentation masks into the sampling process. Specifically, we introduce the SAM-Noise Module, which refines Gaussian noise using segmentation masks to preserve spatial and semantic features. Furthermore, we develop a pixel-wise sampling strategy that dynamically adjusts the residual transfer rate and noise strength based on pixel-level semantic weights, prioritizing semantically rich regions during the diffusion process. To enhance model training, we also propose a semantic consistency loss, which aligns pixel-wise semantic weights between predictions and ground truth. Extensive experiments on both real-world and synthetic datasets demonstrate that SAMSR significantly improves perceptual quality and detail recovery, particularly in semantically complex images. Our code is released at https://github.com/Liu-Zihang/SAMSR.

Semantic-Guided Diffusion Model for Single-Step Image Super-Resolution

TL;DR

SAMSR addresses the bottleneck of single-step diffusion SR in semantically complex images by injecting semantic guidance from segmentation masks. It introduces the SAM-Noise Module to produce spatially adaptive noise and a pixel-wise forward process that modulates residual transfer and noise strength via a mask-derived weight . A semantic consistency loss aligns predicted and ground-truth semantic weights to accelerate training and improve region-aware reconstruction. Empirical results on real and synthetic datasets show improved perceptual quality and detail recovery in semantically rich regions, with faster convergence and competitive fidelity metrics.

Abstract

Diffusion-based image super-resolution (SR) methods have demonstrated remarkable performance. Recent advancements have introduced deterministic sampling processes that reduce inference from 15 iterative steps to a single step, thereby significantly improving the inference speed of existing diffusion models. However, their efficiency remains limited when handling complex semantic regions due to the single-step inference. To address this limitation, we propose SAMSR, a semantic-guided diffusion framework that incorporates semantic segmentation masks into the sampling process. Specifically, we introduce the SAM-Noise Module, which refines Gaussian noise using segmentation masks to preserve spatial and semantic features. Furthermore, we develop a pixel-wise sampling strategy that dynamically adjusts the residual transfer rate and noise strength based on pixel-level semantic weights, prioritizing semantically rich regions during the diffusion process. To enhance model training, we also propose a semantic consistency loss, which aligns pixel-wise semantic weights between predictions and ground truth. Extensive experiments on both real-world and synthetic datasets demonstrate that SAMSR significantly improves perceptual quality and detail recovery, particularly in semantically complex images. Our code is released at https://github.com/Liu-Zihang/SAMSR.
Paper Structure (12 sections, 14 equations, 6 figures, 5 tables, 1 algorithm)

This paper contains 12 sections, 14 equations, 6 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Overview
  5. SAM-Noise Module
  6. SAM-based Forward and Reverse Process
  7. Semantic Consistency Loss
  8. Experiments
  9. Experimental Setup
  10. Experimental Results
  11. Model Analysis
  12. Conclusion

Figures (6)

  • Figure 1: A comparison between the most recent SOTA one-step SR model and our SAMSR model. Different from recent works with simple noise distribution, the proposed method incorporates semantic segmentation information into the noise distribution and gause diffusion process.
  • Figure 3: Architecture of the SAM-Noise Module. The module consists of two main components: (1) A Semantic Enhancement Block integrating bicubic interpolation (BIC), Segment Anything Model (SAM), global average pooling (GAP), and thresholding (TSD) operations for mask generation; (2) A noise sampling and normalization pipeline that leverages semantic information to produce spatially-adaptive noise distributions. This design enables semantically-guided noise generation while preserving structural consistency.
  • Figure 4: Qualitative comparisons on real-world examples. Please zoom in for a better view.
  • Figure : (a) Forward process
  • Figure : (a) Forward process
  • ...and 1 more figures