Next-Scale Prediction: A Self-Supervised Approach for Real-World Image Denoising

TL;DR

NSP introduces Next-Scale Prediction, a self-supervised framework that decouples noise decorrelation from detail preservation for real-world image denoising. By constructing cross-scale training pairs from large PD downsampling, a BSN learns to denoise at a coarse scale while predicting higher-scale targets to recover fine details, achieving state-of-the-art self-supervised performance and enabling by-product super-resolution. A data-pair construction strategy preserves spatial structure and blocks cross-scale noise correlations, with a lightweight BSN modification to support scale transformation. Extensive experiments on real-world benchmarks demonstrate superior denoising quality and reliable restoration of details, underscoring NSP's practical impact for real-world imaging pipelines and potential for high-fidelity SR without retraining.

Abstract

Self-supervised real-world image denoising remains a fundamental challenge, arising from the antagonistic trade-off between decorrelating spatially structured noise and preserving high-frequency details. Existing blind-spot network (BSN) methods rely on pixel-shuffle downsampling (PD) to decorrelate noise, but aggressive downsampling fragments fine structures, while milder downsampling fails to remove correlated noise. To address this, we introduce Next-Scale Prediction (NSP), a novel self-supervised paradigm that decouples noise decorrelation from detail preservation. NSP constructs cross-scale training pairs, where BSN takes low-resolution, fully decorrelated sub-images as input to predict high-resolution targets that retain fine details. As a by-product, NSP naturally supports super-resolution of noisy images without retraining or modification. Extensive experiments demonstrate that NSP achieves state-of-the-art self-supervised denoising performance on real-world benchmarks, significantly alleviating the long-standing conflict between noise decorrelation and detail preservation.

Figures (5)

• Figure 1: Illustration of different training-pair construction strategies for BSN. Inspired by visual autoregressive modeling, our NSP method constructs lower-scale inputs for noise decorrelation and high-scale targets for detail preservation.
• Figure 2: Framework of the proposed NSP. It is built on the PD-based BSN paradigm but reformulates denoising as a next-scale prediction task. The BSN first processes sub-images generated with a large PD factor and then learns to predict their higher-resolution counterparts corresponding to a smaller PD factor. In this way, NSP explicitly decouples the objectives of noise decorrelation and detail preservation.
• Figure 3: Alternative strategies for high-scale target construction. (a) Randomly select $t^2$ pixels. (b) Randomly select $t^2$ pixels and sort in a row-major order. (c) Select the pixels at the intersections of $t$ random rows and columns. (d) Select a consecutive $t\times t$ patch. The comparison between those strategies can be found in Tab. \ref{['tab_strtegy_n']}.
• Figure 4: Qualitative comparisons of image denoising on SIDD Validation.
• Figure 5: Qualitative comparison of image denoising and super-resolution on SIDD Validation.