SEE: See Everything Every Time -- Adaptive Brightness Adjustment for Broad Light Range Images via Events

TL;DR

This work tackles brightness adjustment across a broad range of illumination by leveraging event cameras. It defines a new task and dataset (SEE-600K) and proposes SEE-Net, a compact framework that uses event-aware cross-attention and a brightness prompt to produce pixel-level brightness adjustments, forming a Broad Light-Range representation (BLR). Evaluations on SDE and SEE-600K show SEE-Net achieving state-of-the-art results with strong robustness and denoising, while remaining lightweight at $1.9$M parameters. The approach offers flexible inference via tunable brightness prompts and opens avenues for post-processing and new imaging applications in challenging lighting conditions.

Abstract

Event cameras, with a high dynamic range exceeding $120dB$, significantly outperform traditional embedded cameras, robustly recording detailed changing information under various lighting conditions, including both low- and high-light situations. However, recent research on utilizing event data has primarily focused on low-light image enhancement, neglecting image enhancement and brightness adjustment across a broader range of lighting conditions, such as normal or high illumination. Based on this, we propose a novel research question: how to employ events to enhance and adaptively adjust the brightness of images captured under broad lighting conditions? To investigate this question, we first collected a new dataset, SEE-600K, consisting of 610,126 images and corresponding events across 202 scenarios, each featuring an average of four lighting conditions with over a 1000-fold variation in illumination. Subsequently, we propose a framework that effectively utilizes events to smoothly adjust image brightness through the use of prompts. Our framework captures color through sensor patterns, uses cross-attention to model events as a brightness dictionary, and adjusts the image's dynamic range to form a broad light-range representation (BLR), which is then decoded at the pixel level based on the brightness prompt. Experimental results demonstrate that our method not only performs well on the low-light enhancement dataset but also shows robust performance on broader light-range image enhancement using the SEE-600K dataset. Additionally, our approach enables pixel-level brightness adjustment, providing flexibility for post-processing and inspiring more imaging applications. The dataset and source code are publicly available at: https://github.com/yunfanLu/SEE.

Figures (11)

• Figure 1: (a) and (b): Brightness distributions of the SDE dataset (0$\sim$0.45, low to normal light) and our SEE-600K dataset (0$\sim$1, a broader light range). (c): Previous methods liang2023coherentliang2024towards directly map low-light images to normal-light images. (d): Our SEENet accepts inputs across a broader brightness range and adjusts output brightness through prompts. $f_b$ refers to the function that calculates the brightness of an image.
• Figure 2: (a) data collection setup: Universal Robots UR5e arm replicates precise trajectories with an error margin of $0.03 mm$. (b) IMU data registration:b (1) shows unregistered IMU data, while b (2) displays registered data after timestamp alignment. (c) EVS outputs with different filters:f1 to f4 demonstrate the different ND filters, depicting various lighting levels.
• Figure 3: (a) registration process: Illustration of the multi-level registration process, showing how trajectories, $S$ and $T$, at various levels are iteratively aligned. (b) two trajectories: The example of two aligned images captured along two trajectories. (c) pixel distance change: Temporal distance of pixel between two registered videos, showing a mean alignment error of 0.2957 pixels over time.
• Figure 4: Visualization of different lighting conditions in SEE-600K. We present three scenes captured in SEE-600K, demonstrating its broad illumination coverage. Each row corresponds to a different lighting condition within the same scene: (a) High-Light, (b) Normal-Light, and (c) Low-Light. The columns (I), (II), and (III) represent different environments. Each sub-image pair consists of a frame (left) and its corresponding events (right), highlighting the ability of event cameras to capture fine-grained temporal changes across varying lighting conditions.
• Figure 5: Visualization of scene diversity in SEE-600K. The left side (a) presents 12 representative scenes from SEE-600K, showcasing the variety of environments captured in our dataset, including urban areas, buildings, natural scenes, industrial settings, and structured indoor spaces. With a total of 202 distinct scenes, SEE-600K provides a broad range of lighting conditions and structural diversity for event-based imaging research. The right side (b) displays a word cloud generated from our dataset, illustrating the richness of scene categories and objects present in SEE-600K, further emphasizing its comprehensiveness.