Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Specularity Factorization for Low-Light Enhancement

Saurabh Saini, P J Narayanan

TL;DR

This work tackles zero-reference low-light enhancement by modeling a scene as a sum of multiple additive specular components. It introduces RSFNet, a lightweight, model-driven network that unrolls a sparsity-promoting optimization to estimate K specular factors with three learnable scalars per iteration, totaling about 200 parameters, followed by a fusion network to produce the final image. The specular factors can be fused for enhancement, relighting, or used as priors for tasks like dehazing, deraining, and deblurring, demonstrating strong state-of-the-art performance and good generalization on diverse datasets. The approach offers interpretable intermediate factors and broad applicability across multi-task, multi-domain image enhancement, with code and data released for reproducibility.

Abstract

We present a new additive image factorization technique that treats images to be composed of multiple latent specular components which can be simply estimated recursively by modulating the sparsity during decomposition. Our model-driven {\em RSFNet} estimates these factors by unrolling the optimization into network layers requiring only a few scalars to be learned. The resultant factors are interpretable by design and can be fused for different image enhancement tasks via a network or combined directly by the user in a controllable fashion. Based on RSFNet, we detail a zero-reference Low Light Enhancement (LLE) application trained without paired or unpaired supervision. Our system improves the state-of-the-art performance on standard benchmarks and achieves better generalization on multiple other datasets. We also integrate our factors with other task specific fusion networks for applications like deraining, deblurring and dehazing with negligible overhead thereby highlighting the multi-domain and multi-task generalizability of our proposed RSFNet. The code and data is released for reproducibility on the project homepage.

Specularity Factorization for Low-Light Enhancement

TL;DR

This work tackles zero-reference low-light enhancement by modeling a scene as a sum of multiple additive specular components. It introduces RSFNet, a lightweight, model-driven network that unrolls a sparsity-promoting optimization to estimate K specular factors with three learnable scalars per iteration, totaling about 200 parameters, followed by a fusion network to produce the final image. The specular factors can be fused for enhancement, relighting, or used as priors for tasks like dehazing, deraining, and deblurring, demonstrating strong state-of-the-art performance and good generalization on diverse datasets. The approach offers interpretable intermediate factors and broad applicability across multi-task, multi-domain image enhancement, with code and data released for reproducibility.

Abstract

We present a new additive image factorization technique that treats images to be composed of multiple latent specular components which can be simply estimated recursively by modulating the sparsity during decomposition. Our model-driven {\em RSFNet} estimates these factors by unrolling the optimization into network layers requiring only a few scalars to be learned. The resultant factors are interpretable by design and can be fused for different image enhancement tasks via a network or combined directly by the user in a controllable fashion. Based on RSFNet, we detail a zero-reference Low Light Enhancement (LLE) application trained without paired or unpaired supervision. Our system improves the state-of-the-art performance on standard benchmarks and achieves better generalization on multiple other datasets. We also integrate our factors with other task specific fusion networks for applications like deraining, deblurring and dehazing with negligible overhead thereby highlighting the multi-domain and multi-task generalizability of our proposed RSFNet. The code and data is released for reproducibility on the project homepage.
Paper Structure (15 sections, 22 equations, 19 figures, 11 tables, 1 algorithm)

This paper contains 15 sections, 22 equations, 19 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Model-based LLE:
  4. Data-driven LLE:
  5. Model-Driven Networks:
  6. Retinex Factorization for LLE:
  7. Others Factorization Strategies:
  8. Approach
  9. Outline:
  10. Factorization Network: RSFNet
  11. Specularity Estimation:
  12. Fusion Network
  13. Experiments and Results
  14. Conclusions
  15. Ethical Concerns:

Figures (19)

  • Figure 1: Sepcularity Factorization: We factorize a single input image (blue box, top row) into multiple soft specular factors (rescaled for visualization) based on their similar illumination characteristics (note table shadow and lamp reflection). Our factors directly enable zero-reference low-light enhancement and user controlled image relighting (bottom left). Additionally, they can also be used as a plug-and-play prior for various supervised image enhancement tasks like dehazing, deraining and deblurring. On right, our conceptual block diagram.
  • Figure 2: Categorization and Motivation: Left shows categorization of various LLE solution types (\ref{['sec:relatedwork']}). Middle plot shows the relationship between five factor cluster centers w.r.t each other and the background comprising of shadow/non-shadow regions estimated using PCA dimensionality reduced DINO features DINO. Gradual progression of feature cluster centers from highlight region to shadow region indicates their capability to capture various illumination regions in an image. Top right shows one data point from CHUK dataset CHUK with mask, processed shadow/highlight regions and extracted factors. Bottom right plots distinguish our specular fuzzy factors from intensity thresholding based binary division, with ours allowing more diverse distributions and richer representation.
  • Figure 3: Block Diagram: Our factorization module (RSFNet) splits a given image into multiple specular components using model-driven unrolled optimization. Then fusion module combines all the factors to generate the enhanced output.
  • Figure 4: Results: Qualitative comparison of our method (green box) with other solutions (from top left 3 per row: SDD SDD, ECNet ExcNet, ZDCE zeroDCE; ZD++ zeroDCEPP, RUAS RUAS, SCI SCI; PNet PSENet, GDP GDP and our RSFNet respectively). Our method generate natural looking images by handling noisy over and under exposed regions equally well, without over-saturating color or losing geometric details.
  • Figure 5: Analysis: On left, our average score on all datasets vs. other methods (more area implies better). On right, ablation analysis with varying number of factors.
  • ...and 14 more figures