Pixel to Gaussian: Ultra-Fast Continuous Super-Resolution with 2D Gaussian Modeling

TL;DR

ContinuousSR reframes ASSR as direct reconstruction of a continuous HR signal from LR inputs using 2D Gaussian modeling. It replaces costly upsampling and decoding with Gaussian Splatting to render arbitrary scales in milliseconds. The approach identifies a Deep Gaussian Prior to guide covariance and kernel design and introduces Adaptive Position Drifting and Color Gaussian Mapping to optimize kernel placement and color. On seven benchmarks, it achieves state-of-the-art reconstruction quality and ultra-fast inference, including up to 19.5x speedups and up to 0.9 dB PSNR gains over prior methods, indicating strong practical potential for real-time ASSR.

Abstract

Arbitrary-scale super-resolution (ASSR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs with arbitrary upsampling factors using a single model, addressing the limitations of traditional SR methods constrained to fixed-scale factors (\textit{e.g.}, $\times$ 2). Recent advances leveraging implicit neural representation (INR) have achieved great progress by modeling coordinate-to-pixel mappings. However, the efficiency of these methods may suffer from repeated upsampling and decoding, while their reconstruction fidelity and quality are constrained by the intrinsic representational limitations of coordinate-based functions. To address these challenges, we propose a novel ContinuousSR framework with a Pixel-to-Gaussian paradigm, which explicitly reconstructs 2D continuous HR signals from LR images using Gaussian Splatting. This approach eliminates the need for time-consuming upsampling and decoding, enabling extremely fast arbitrary-scale super-resolution. Once the Gaussian field is built in a single pass, ContinuousSR can perform arbitrary-scale rendering in just 1ms per scale. Our method introduces several key innovations. Through statistical ana

Figures (9)

• Figure 1: (a) Leveraging implicit modeling, existing ASSR methods rely on multiple upsampling and decoding steps to reconstruct HR images at different scales, which leads to low efficiency and performance. (b-d) Our method explicitly reconstructs 2D continuous HR signals from LR images in a single pass. Then, fast rendering replaces the time-consuming upsampling and decoding process to reconstruct HR images at different scales, significantly improving both performance (0.90 dB in Manga109) and efficiency (19.5× speedup).
• Figure 2: (a) Directly learning the end-to-end model from LR to the Gaussian field is challenging due to the vastness and sensitivity of the Gaussian space. (b-c) Through statistical analysis of 40,000 natural images, we uncover the Deep Gaussian Prior and propose Position Drifting, Covariance Prior, and Color Mapping to propose a novel ContinuousSR, enhancing the quality of the Gaussian field.
• Figure 3: An overview of the proposed ContinuousSR framework, which consists of three key innovations: DGP-Driven Covariance Weighting (DDCW), Adaptive Position Drifting (APD), and Color Gaussian Mapping (CGM).
• Figure 4: Qualitative comparison. The visual quality of our method outperforms existing methods. Please zoom in for a better view.
• Figure 5: User study.