Learning to Control Camera Exposure via Reinforcement Learning

TL;DR

The paper tackles robust camera exposure control under dynamic lighting, a critical bottleneck for vision systems. It introduces DRL-AE, a deep reinforcement learning framework that jointly optimizes exposure time and gain using a lightweight vectorized RoI intensity history, continuous-relative actions, a flicker-aware reward, static-to-dynamic curriculum, and domain randomization. Empirical results show rapid convergence within about five steps and real-time CPU processing (~1 ms inference, ~6 ms total per frame), with superior feature extraction and object detection performance over built-in AE across darkroom, dataset, and real-world driving scenarios. The work demonstrates that DRL can provide fast, generalizable exposure control that enhances downstream perception tasks in diverse lighting, marking a first step toward robust, perception-aware camera control.

Abstract

Adjusting camera exposure in arbitrary lighting conditions is the first step to ensure the functionality of computer vision applications. Poorly adjusted camera exposure often leads to critical failure and performance degradation. Traditional camera exposure control methods require multiple convergence steps and time-consuming processes, making them unsuitable for dynamic lighting conditions. In this paper, we propose a new camera exposure control framework that rapidly controls camera exposure while performing real-time processing by exploiting deep reinforcement learning. The proposed framework consists of four contributions: 1) a simplified training ground to simulate real-world's diverse and dynamic lighting changes, 2) flickering and image attribute-aware reward design, along with lightweight state design for real-time processing, 3) a static-to-dynamic lighting curriculum to gradually improve the agent's exposure-adjusting capability, and 4) domain randomization techniques to alleviate the limitation of the training ground and achieve seamless generalization in the wild.As a result, our proposed method rapidly reaches a desired exposure level within five steps with real-time processing (1 ms). Also, the acquired images are well-exposed and show superiority in various computer vision tasks, such as feature extraction and object detection.

Figures (11)

• Figure 1: Automatic camera exposure control via deep reinforcement learning. Our proposed method, named DRL-AE, trains an agent to control camera exposure parameters (i.e., exposure time and gain) to acquire well-exposed images with rapid convergence and real-time processing (1ms on a CPU device). The trained agent instantly converges within five frames under dramatic lighting change scenario (a) and affects the performance of various vision applications (b), compared to the camera built-in AE controller muramatsu1997photometrysampat1999system.
• Figure 2: Training framework overview. Our DRL agent is trained with the SAC algorithm in the light-controlled dark room environment. For each episode, a lighting condition is assigned by the current curriculum level. The lighting condition can be fixed at random brightness or dynamically changed within each episode, depending on the level. Given the lighting condition, the agent takes a vectorized intensity history for a randomly selected RoI patch as a state. Afterward, the agent estimates exposure time and gain differences that maximize a reward function. With this framework, the trained agent successfully generalized into a real environment without additional training.
• Figure 3: Convergent step comparison in exposure control dataset shin2019camera. Within three frames, our method already reaches a well-exposed image (a) with minimum exploration (b). On the other hand, Shin et al. shin2019camera search local areas with multiple steps (about 30 frames) to converge.
• Figure 4: Real-world generalization. We compare our method with the camera's built-in exposure control algorithm in real-world scenarios. Camera lenses are occluded at the initial and suddenly removed in the first frame. Our agent converges to a well-exposed image within 3-5 frames. Yet, the built-in AE algorithm is still in the middle of adjusting the exposure parameters and is far from the well-exposed image, especially in the indoor case. Note that our agent is only trained in the light-controlled darkroom, and this is the zero-shot inference result in the wild.
• Figure 5: SIFT lowe1999object feature extraction result. Captured images from the proposed algorithm and built-in AE are processed to detect SIFT features. The images were simultaneously captured in real-time from two separate cameras equipped on a driving vehicle. Our method can provide plenty of SIFT features over the image plane. On average, our method detects 38% more features across a total of 5355 images.