Gradient as Conditions: Rethinking HOG for All-in-one Image Restoration

TL;DR

This work tackles all-in-one image restoration by introducing HOGformer, a Transformer-based network that uses learnable HOG gradient descriptors as explicit degradation priors. It blends Dynamic HOG-aware Self-Attention (DHOGSA) with a Dynamic Interaction Feed-Forward (DIFF) and a dedicated HOG loss to improve structure, edges, and cross-degradation generalization. Extensive experiments across adverse weather and natural degradations demonstrate state-of-the-art performance and strong generalization, while ablations validate the utility of LDRConv, BHOGR/FHOGR, and HOG-guided mechanisms. The approach offers a principled, efficient alternative to implicit conditioning for robust AIR in real-world scenarios.

Abstract

All-in-one image restoration (AIR) aims to address diverse degradations within a unified model by leveraging informative degradation conditions to guide the restoration process. However, existing methods often rely on implicitly learned priors, which may entangle feature representations and hinder performance in complex or unseen scenarios. Histogram of Oriented Gradients (HOG) as a classical gradient representation, we observe that it has strong discriminative capability across diverse degradations, making it a powerful and interpretable prior for AIR. Based on this insight, we propose HOGformer, a Transformer-based model that integrates learnable HOG features for degradation-aware restoration. The core of HOGformer is a Dynamic HOG-aware Self-Attention (DHOGSA) mechanism, which adaptively models long-range spatial dependencies conditioned on degradation-specific cues encoded by HOG descriptors. To further adapt the heterogeneity of degradations in AIR, we propose a Dynamic Interaction Feed-Forward (DIFF) module that facilitates channel-spatial interactions, enabling robust feature transformation under diverse degradations. Besides, we propose a HOG loss to explicitly enhance structural fidelity and edge sharpness. Extensive experiments on a variety of benchmarks, including adverse weather and natural degradations, demonstrate that HOGformer achieves state-of-the-art performance and generalizes well to complex real-world scenarios.Code is available at https://github.com/Fire-friend/HOGformer.

Figures (7)

• Figure 1: Visualization of HOG feature distributions under various degradations. (a) Example images of different weather conditions with corresponding HOG feature visualizations. (b) HOG features for five natural degradations zheng2024selective and three adverse weather degradations sun2024restoring, using 100 randomly selected images for each degradation.
• Figure 2: The overall architecture of our HOGformer. It includes the HOG Transformer block with the Dynamic HOG-aware Self-Attention (DHOGSA) module and Dynamic Interaction Feed-Forward (DIFF) module.
• Figure 3: HOG-guided mechanism: (a) Patch-level sorting. (b) Pixel-level sorting. (c) HOG feature extraction process.
• Figure 4: Visual comparison for deraining sun2024restoringozdenizci2023restoring. Zoom in for the best visualization.
• Figure 5: Comparisons on multiple degradations (rain-fog-snow). Top: PSNR and SSIM comparisons between Histformer and our method. Bottom: Visual results on a representative example.