Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pure-Pass: Fine-Grained, Adaptive Masking for Dynamic Token-Mixing Routing in Lightweight Image Super-Resolution

Junyu Wu, Jie Liu, Jie Tang, Gangshan Wu

TL;DR

This work presents Pure-Pass (PP), a pixel-level masking mechanism that adaptively bypasses texture-deficient pixels in lightweight image super-resolution. PP uses a fixed set of color centers to classify pixels, then applies window-based and cross-shift fusion to produce fine-grained masks, paired with an information-preserving compensation path to keep fidelity. Integrated into the ATD-light architecture as PP-ATD-light, PP achieves comparable or superior reconstruction quality while reducing FLOPs and maintaining a small parameter footprint relative to CAMixer variants. The approach demonstrates strong performance gains on standard SR benchmarks, notably Urban100, and offers practical benefits for deploying efficient SR models in resource-constrained settings, with only negligible overhead. Mathematical notation is used to define the masking and compensation operations, underscoring the method's rigor and reproducibility.

Abstract

Image Super-Resolution (SR) aims to reconstruct high-resolution images from low-resolution counterparts, but the computational complexity of deep learning-based methods often hinders practical deployment. CAMixer is the pioneering work to integrate the advantages of existing lightweight SR methods and proposes a content-aware mixer to route token mixers of varied complexities according to the difficulty of content recovery. However, several limitations remain, such as poor adaptability, coarse-grained masking and spatial inflexibility, among others. We propose Pure-Pass (PP), a pixel-level masking mechanism that identifies pure pixels and exempts them from expensive computations. PP utilizes fixed color center points to classify pixels into distinct categories, enabling fine-grained, spatially flexible masking while maintaining adaptive flexibility. Integrated into the state-of-the-art ATD-light model, PP-ATD-light achieves superior SR performance with minimal overhead, outperforming CAMixer-ATD-light in reconstruction quality and parameter efficiency when saving a similar amount of computation.

Pure-Pass: Fine-Grained, Adaptive Masking for Dynamic Token-Mixing Routing in Lightweight Image Super-Resolution

TL;DR

This work presents Pure-Pass (PP), a pixel-level masking mechanism that adaptively bypasses texture-deficient pixels in lightweight image super-resolution. PP uses a fixed set of color centers to classify pixels, then applies window-based and cross-shift fusion to produce fine-grained masks, paired with an information-preserving compensation path to keep fidelity. Integrated into the ATD-light architecture as PP-ATD-light, PP achieves comparable or superior reconstruction quality while reducing FLOPs and maintaining a small parameter footprint relative to CAMixer variants. The approach demonstrates strong performance gains on standard SR benchmarks, notably Urban100, and offers practical benefits for deploying efficient SR models in resource-constrained settings, with only negligible overhead. Mathematical notation is used to define the masking and compensation operations, underscoring the method's rigor and reproducibility.

Abstract

Image Super-Resolution (SR) aims to reconstruct high-resolution images from low-resolution counterparts, but the computational complexity of deep learning-based methods often hinders practical deployment. CAMixer is the pioneering work to integrate the advantages of existing lightweight SR methods and proposes a content-aware mixer to route token mixers of varied complexities according to the difficulty of content recovery. However, several limitations remain, such as poor adaptability, coarse-grained masking and spatial inflexibility, among others. We propose Pure-Pass (PP), a pixel-level masking mechanism that identifies pure pixels and exempts them from expensive computations. PP utilizes fixed color center points to classify pixels into distinct categories, enabling fine-grained, spatially flexible masking while maintaining adaptive flexibility. Integrated into the state-of-the-art ATD-light model, PP-ATD-light achieves superior SR performance with minimal overhead, outperforming CAMixer-ATD-light in reconstruction quality and parameter efficiency when saving a similar amount of computation.

Paper Structure

This paper contains 23 sections, 11 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Accelerating Framework for SR
  4. Dynamic Token-Mixing Routing
  5. Method
  6. Pure-Pass: Pixel-level Mask Generation
  7. Pixel Classification
  8. Window-based Pixel-level Mask Generation
  9. Cross-Shift Mask Fusion
  10. PP-ATD-light: Pure-Pass Accelerated ATD-light
  11. Original AC-MSA Procedure
  12. Pure-Pass Optimized AC-MSA
  13. Information-Preserving Compensation
  14. Discussion: Advantages
  15. Experiment
  16. ...and 8 more sections

Figures (3)

  • Figure 1: An example of localized regions containing complex textures positioned at the intersection points of four adjacent windows within a cross-shaped configuration, while being surrounded by homogeneous, textureless areas.
  • Figure 2: Pure-Pass mask generation pipeline: (a) Pixel Classification, (b) Window-based Pixel-level Mask Generation, (c) Cross-Shift Mask Fusion
  • Figure 3: Masking Comparison. The white masks in (b), (c), (e), (f) represent areas considered easy for SR, where computational overhead will be saved.