Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CLIP-Guided Unsupervised Semantic-Aware Exposure Correction

Puzhen Wu, Han Weng, Quan Zheng, Yi Zhan, Hewei Wang, Yiming Li, Jiahui Han, Rui Xu

TL;DR

Real-world exposure correction is hampered by region-wise semantic variation and the lack of ground-truth labels. The authors propose an unsupervised semantic-aware exposure correction network that fuses FastSAM-derived semantic features into image representations via adaptive fusion in multi-scale SIMR blocks, complemented by a CLIP-guided pseudo-ground-truth generator and a semantic-prompt consistency loss. The three main contributions are a semantics-informed reconstruction network, automated pseudo-GT generation, and a loss enforcing semantic consistency and image-prompt alignment. Experiments on real-world datasets demonstrate state-of-the-art performance among unsupervised methods in both quantitative metrics and visual quality.

Abstract

Improper exposure often leads to severe loss of details, color distortion, and reduced contrast. Exposure correction still faces two critical challenges: (1) the ignorance of object-wise regional semantic information causes the color shift artifacts; (2) real-world exposure images generally have no ground-truth labels, and its labeling entails massive manual editing. To tackle the challenges, we propose a new unsupervised semantic-aware exposure correction network. It contains an adaptive semantic-aware fusion module, which effectively fuses the semantic information extracted from a pre-trained Fast Segment Anything Model into a shared image feature space. Then the fused features are used by our multi-scale residual spatial mamba group to restore the details and adjust the exposure. To avoid manual editing, we propose a pseudo-ground truth generator guided by CLIP, which is fine-tuned to automatically identify exposure situations and instruct the tailored corrections. Also, we leverage the rich priors from the FastSAM and CLIP to develop a semantic-prompt consistency loss to enforce semantic consistency and image-prompt alignment for unsupervised training. Comprehensive experimental results illustrate the effectiveness of our method in correcting real-world exposure images and outperforms state-of-the-art unsupervised methods both numerically and visually.

CLIP-Guided Unsupervised Semantic-Aware Exposure Correction

TL;DR

Real-world exposure correction is hampered by region-wise semantic variation and the lack of ground-truth labels. The authors propose an unsupervised semantic-aware exposure correction network that fuses FastSAM-derived semantic features into image representations via adaptive fusion in multi-scale SIMR blocks, complemented by a CLIP-guided pseudo-ground-truth generator and a semantic-prompt consistency loss. The three main contributions are a semantics-informed reconstruction network, automated pseudo-GT generation, and a loss enforcing semantic consistency and image-prompt alignment. Experiments on real-world datasets demonstrate state-of-the-art performance among unsupervised methods in both quantitative metrics and visual quality.

Abstract

Improper exposure often leads to severe loss of details, color distortion, and reduced contrast. Exposure correction still faces two critical challenges: (1) the ignorance of object-wise regional semantic information causes the color shift artifacts; (2) real-world exposure images generally have no ground-truth labels, and its labeling entails massive manual editing. To tackle the challenges, we propose a new unsupervised semantic-aware exposure correction network. It contains an adaptive semantic-aware fusion module, which effectively fuses the semantic information extracted from a pre-trained Fast Segment Anything Model into a shared image feature space. Then the fused features are used by our multi-scale residual spatial mamba group to restore the details and adjust the exposure. To avoid manual editing, we propose a pseudo-ground truth generator guided by CLIP, which is fine-tuned to automatically identify exposure situations and instruct the tailored corrections. Also, we leverage the rich priors from the FastSAM and CLIP to develop a semantic-prompt consistency loss to enforce semantic consistency and image-prompt alignment for unsupervised training. Comprehensive experimental results illustrate the effectiveness of our method in correcting real-world exposure images and outperforms state-of-the-art unsupervised methods both numerically and visually.
Paper Structure (11 sections, 8 equations, 8 figures, 3 tables)

This paper contains 11 sections, 8 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Semantic-Aware Exposure Correction Network
  5. CLIP Guided Pseudo-GT Generation
  6. Semantic-Prompt Consistency Loss
  7. Experiments
  8. Experimental Setup
  9. Comparisons with State-of-the-art Methods
  10. Ablation Study
  11. Conclusion

Figures (8)

  • Figure 1: Performance comparison with other unsupervised methods.
  • Figure 2: Overview of our framework: (a) The semantic-aware exposure correction network. (b) Pseudo-ground truth generator. (c) Loss.
  • Figure 3: FastSAM fastsam segmentation results.
  • Figure 4: The architecture of the ASF module.
  • Figure 5: Qualitative comparison with state-of-the-art unsupervised methods.
  • ...and 3 more figures