Temporal Brightness Management for Immersive Content

TL;DR

This work tackles power efficiency in VR by introducing a content-aware temporal brightness modulation framework that aims to meet a target power budget while preserving uniform temporal contrast loss. It combines a perceptual luminance-contrast model based on a multi-band Laplacian pyramid, CSF weighting, and a Zeng transducer masking approach to quantify visible contrast, and then optimizes per-frame brightness factors $b_i$ under a power constraint and smoothness requirements. The offline optimization is complemented by perceptual experiments, a hardware proxy for real power savings, and a real-time PID-based online extension, collectively demonstrating improved perceived detail and energy savings over uniform dimming. The approach emphasizes low-level vision compatibility and VR-specific perceptual characteristics, suggesting broad applicability to power-aware rendering and potential integration with future low-level vision models for real-time content-aware brightness management.

Abstract

Modern virtual reality headsets demand significant computational resources to render high-resolution content in real-time. Therefore, prioritizing power efficiency becomes crucial, particularly for portable versions reliant on batteries. A significant portion of the energy consumed by these systems is attributed to their displays. Dimming the screen can save a considerable amount of energy; however, it may also result in a loss of visible details and contrast in the displayed content. While contrast may be partially restored by applying post-processing contrast enhancement steps, our work is orthogonal to these approaches, and focuses on optimal temporal modulation of screen brightness. We propose a technique that modulates brightness over time while minimizing the potential loss of visible details and avoiding noticeable temporal instability. Given a predetermined power budget and a video sequence, we achieve this by measuring contrast loss through band decomposition of the luminance image and optimizing the brightness level of each frame offline to ensure uniform temporal contrast loss. We evaluate our method through a series of subjective experiments and an ablation study, on a variety of content. We showcase its power-saving capabilities in practice using a built-in hardware proxy. Finally, we present an online version of our approach which further emphasizes the potential for low level vision models to be leveraged in power saving settings to preserve content quality.

Figures (11)

• Figure 1: Pipeline of our method. The conversion to luminance is achieved using the luminance curve of the display and applying the display model from Mantiuk et al. (Equation \ref{['eq:display-model']}). The self-parameters for masking remain consistent with the original model proposed by Zeng et al. zeng2000point: $\alpha = 0.7$ and $\beta = 0.2$. The output value $C_v$ represents the visible contrast in the frame.
• Figure 2: Correlation between luminance and the slope at the visibility threshold (probability of 75% of detecting the change). The curve is a fourth-degree polynomial fitted to the data points.
• Figure 3: Visible contrast (white) at the highest frequency level of the band decomposition after binary thresholding, and regions where contrast loss occurs between the dimmed frame and the original frame (red), for target average brightness levels of 0.4 (top) and 0.8 (middle). The original frame is displayed at the bottom of the image. The lower the backlight brightness, the higher the amount of detail lost.
• Figure 4: We apply our method to a sequence of checkerboard patterns (top, input frames), where the white squares gradually transition to black through various shades of grey. Our method compensates for the increasing loss of visible contrast by enhancing the brightness accordingly.
• Figure 5: Time-lapse sequence capturing the Sun's movement (top, input frames). Our technique optimizes power allocation by spending more of the "brightness budget" during the initial and final parts of the video, where the frames are darker. At the same time, during the middle section, the dimming can be more aggressive.