Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Histogram Assisted Quality Aware Generative Model for Resolution Invariant NIR Image Colorization

Abhinav Attri, Rajeev Ranjan Dwivedi, Samiran Das, Vinod Kumar Kurmi

TL;DR

HAQAGen addresses the challenging NIR-to-RGB translation problem by unifying global color statistics with local chromatic priors and texture-aware supervision within a Mamba-based backbone. It employs a dual-branch generator conditioned by SPADE using HSV priors, and a reconstruction objective that combines frozen texture features, perceptual semantics, and a differentiable histogram (CDF) loss to enforce color realism and texture fidelity. The method also introduces adaptive-resolution inference through patch-based sliding-window processing and seam-aware blending, enabling native-resolution colorization without texture loss. Across four diverse datasets, HAQAGen delivers improved perceptual quality (LPIPS), chromatic fidelity (AE), and competitive fidelity/structure metrics, while maintaining generalization and scalability for high-resolution imagery. These advances position HAQAGen as a practical and robust solution for NIR-to-RGB translation in real-world surveillance, remote sensing, and low-light perception tasks.

Abstract

We present HAQAGen, a unified generative model for resolution-invariant NIR-to-RGB colorization that balances chromatic realism with structural fidelity. The proposed model introduces (i) a combined loss term aligning the global color statistics through differentiable histogram matching, perceptual image quality measure, and feature based similarity to preserve texture information, (ii) local hue-saturation priors injected via Spatially Adaptive Denormalization (SPADE) to stabilize chromatic reconstruction, and (iii) texture-aware supervision within a Mamba backbone to preserve fine details. We introduce an adaptive-resolution inference engine that further enables high-resolution translation without sacrificing quality. Our proposed NIR-to-RGB translation model simultaneously enforces global color statistics and local chromatic consistency, while scaling to native resolutions without compromising texture fidelity or generalization. Extensive evaluations on FANVID, OMSIV, VCIP2020, and RGB2NIR using different evaluation metrics demonstrate consistent improvements over state-of-the-art baseline methods. HAQAGen produces images with sharper textures, natural colors, attaining significant gains as per perceptual metrics. These results position HAQAGen as a scalable and effective solution for NIR-to-RGB translation across diverse imaging scenarios. Project Page: https://rajeev-dw9.github.io/HAQAGen/

Histogram Assisted Quality Aware Generative Model for Resolution Invariant NIR Image Colorization

TL;DR

HAQAGen addresses the challenging NIR-to-RGB translation problem by unifying global color statistics with local chromatic priors and texture-aware supervision within a Mamba-based backbone. It employs a dual-branch generator conditioned by SPADE using HSV priors, and a reconstruction objective that combines frozen texture features, perceptual semantics, and a differentiable histogram (CDF) loss to enforce color realism and texture fidelity. The method also introduces adaptive-resolution inference through patch-based sliding-window processing and seam-aware blending, enabling native-resolution colorization without texture loss. Across four diverse datasets, HAQAGen delivers improved perceptual quality (LPIPS), chromatic fidelity (AE), and competitive fidelity/structure metrics, while maintaining generalization and scalability for high-resolution imagery. These advances position HAQAGen as a practical and robust solution for NIR-to-RGB translation in real-world surveillance, remote sensing, and low-light perception tasks.

Abstract

We present HAQAGen, a unified generative model for resolution-invariant NIR-to-RGB colorization that balances chromatic realism with structural fidelity. The proposed model introduces (i) a combined loss term aligning the global color statistics through differentiable histogram matching, perceptual image quality measure, and feature based similarity to preserve texture information, (ii) local hue-saturation priors injected via Spatially Adaptive Denormalization (SPADE) to stabilize chromatic reconstruction, and (iii) texture-aware supervision within a Mamba backbone to preserve fine details. We introduce an adaptive-resolution inference engine that further enables high-resolution translation without sacrificing quality. Our proposed NIR-to-RGB translation model simultaneously enforces global color statistics and local chromatic consistency, while scaling to native resolutions without compromising texture fidelity or generalization. Extensive evaluations on FANVID, OMSIV, VCIP2020, and RGB2NIR using different evaluation metrics demonstrate consistent improvements over state-of-the-art baseline methods. HAQAGen produces images with sharper textures, natural colors, attaining significant gains as per perceptual metrics. These results position HAQAGen as a scalable and effective solution for NIR-to-RGB translation across diverse imaging scenarios. Project Page: https://rajeev-dw9.github.io/HAQAGen/
Paper Structure (15 sections, 4 equations, 5 figures, 5 tables, 1 algorithm)

This paper contains 15 sections, 4 equations, 5 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Backbone and Overall Design
  5. Mamba-based encoder–decoder.
  6. Learning Objective
  7. HAQAGen reconstruction tier.
  8. Adaptive-Resolution Inference
  9. Content-aware downscaling (mitigating uniform-patch bias).
  10. Experimental Setup & Implementation Details
  11. Datasets
  12. Training protocol.
  13. Preprocessing and Augmentation
  14. Results and Discussion
  15. Conclusion

Figures (5)

  • Figure 1: Proposed framework. NIR features feed two branches: an HSV Predictor and an RGB Reconstruction network. HSV guides the RGB decoder via SPADE SPADE, with dual discriminators and multi-term losses ensuring realism and consistency.
  • Figure 2: Comparison of FANVID dataset: (1) NIR input, (2) ground-truth RGB, (3) prediction with resizing (blurred), (4) prediction with adaptive resolution (sharper texture, better color).
  • Figure 3: Adaptive patching: stride-based tiling, patch-wise colorization, and feathered stitching for seamless RGB output.
  • Figure 4: Qualitative comparison on the VCIP2020 dataset Yang2023aCoColor: (1) NIR input, (2) DRSformer Chen2023DRSformer, (3) CoColor Yang2023aCoColor, (4) ColorMamba Zhai2024ColorMamb, (5) our proposed HAQAGen, and (6) ground-truth RGB. HAQAGen achieves sharper textures, more natural chromatic distributions, and better structural fidelity compared to prior baselines.
  • Figure 5: OMSIV soria2017rgb-nirDataset: Col. 1 NIR; Col. 2 GT; Col. 3 ColorMamba (resized); Col. 4 HAQAGen (adaptive). Sliding-window inference preserves texture and tone continuity in high-resolution settings, outperforming global resizing.