Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HMANet: Hybrid Multi-Axis Aggregation Network for Image Super-Resolution

Shu-Chuan Chu, Zhi-Chao Dou, Jeng-Shyang Pan, Shaowei Weng, Junbao Li

TL;DR

HMA addresses the limited global context of window-based Transformers in single-image super-resolution by introducing a hybrid architecture that fuses local and global information. It combines Residual Hybrid Transformer Blocks with Grid Attention Blocks, aided by Fused Attention Blocks that merge convolutional and self-attention processing, and a Grid-MSA mechanism to exploit self-similarity across the image. A task-specific pre-training strategy further enhances representation without sacrificing SR performance, yielding state-of-the-art PSNR/SSIM on standard benchmarks and strong qualitative results. The work demonstrates that integrating multi-axis attention and cross-domain feature interactions effectively improves detail fidelity and texture reconstruction in SR, with practical implications for high-quality upscaling in real-world applications.

Abstract

Transformer-based methods have demonstrated excellent performance on super-resolution visual tasks, surpassing conventional convolutional neural networks. However, existing work typically restricts self-attention computation to non-overlapping windows to save computational costs. This means that Transformer-based networks can only use input information from a limited spatial range. Therefore, a novel Hybrid Multi-Axis Aggregation network (HMA) is proposed in this paper to exploit feature potential information better. HMA is constructed by stacking Residual Hybrid Transformer Blocks(RHTB) and Grid Attention Blocks(GAB). On the one side, RHTB combines channel attention and self-attention to enhance non-local feature fusion and produce more attractive visual results. Conversely, GAB is used in cross-domain information interaction to jointly model similar features and obtain a larger perceptual field. For the super-resolution task in the training phase, a novel pre-training method is designed to enhance the model representation capabilities further and validate the proposed model's effectiveness through many experiments. The experimental results show that HMA outperforms the state-of-the-art methods on the benchmark dataset. We provide code and models at https://github.com/korouuuuu/HMA.

HMANet: Hybrid Multi-Axis Aggregation Network for Image Super-Resolution

TL;DR

HMA addresses the limited global context of window-based Transformers in single-image super-resolution by introducing a hybrid architecture that fuses local and global information. It combines Residual Hybrid Transformer Blocks with Grid Attention Blocks, aided by Fused Attention Blocks that merge convolutional and self-attention processing, and a Grid-MSA mechanism to exploit self-similarity across the image. A task-specific pre-training strategy further enhances representation without sacrificing SR performance, yielding state-of-the-art PSNR/SSIM on standard benchmarks and strong qualitative results. The work demonstrates that integrating multi-axis attention and cross-domain feature interactions effectively improves detail fidelity and texture reconstruction in SR, with practical implications for high-quality upscaling in real-world applications.

Abstract

Transformer-based methods have demonstrated excellent performance on super-resolution visual tasks, surpassing conventional convolutional neural networks. However, existing work typically restricts self-attention computation to non-overlapping windows to save computational costs. This means that Transformer-based networks can only use input information from a limited spatial range. Therefore, a novel Hybrid Multi-Axis Aggregation network (HMA) is proposed in this paper to exploit feature potential information better. HMA is constructed by stacking Residual Hybrid Transformer Blocks(RHTB) and Grid Attention Blocks(GAB). On the one side, RHTB combines channel attention and self-attention to enhance non-local feature fusion and produce more attractive visual results. Conversely, GAB is used in cross-domain information interaction to jointly model similar features and obtain a larger perceptual field. For the super-resolution task in the training phase, a novel pre-training method is designed to enhance the model representation capabilities further and validate the proposed model's effectiveness through many experiments. The experimental results show that HMA outperforms the state-of-the-art methods on the benchmark dataset. We provide code and models at https://github.com/korouuuuu/HMA.
Paper Structure (26 sections, 14 equations, 10 figures, 7 tables)

This paper contains 26 sections, 14 equations, 10 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. CNN-Based SISR
  4. Transformer-Based SISR
  5. Self-similarity based image restoration
  6. Motivation
  7. Proposed Method
  8. Overall Architecture
  9. Fused Attention Block (FAB)
  10. Grid Attention Block(GAB)
  11. Pre-training strategy
  12. Experiments
  13. Experimental Setup
  14. Training Details
  15. Ablation Study
  16. ...and 11 more sections

Figures (10)

  • Figure 1: The performance of the proposed HMA is compared with the state-of-the-art SwinIR, ART, HAT, and GRL methods in terms of PSNR(dB). Our method outperforms the state-of-the-art methods by 0.1dB$\sim$1.4dB.
  • Figure 2: Example of image similarity based on non-local textures. Image from DIV2K:0830.
  • Figure 3: Grid Attention Strategies. We divide the feature map into sparse areas at specific intervals ($K=4$) and then compute the self-attention within each set of sparse areas.
  • Figure 4: (a) CKA similarity between all G and Q in the $\times$2 SR model. (b) CKA similarity between all G and K in the $\times$2 SR model.
  • Figure 5: The overall architecture of HMA and the structure of RHTB and GAB.
  • ...and 5 more figures