It's Not Just a Phase: Creating Phase-Aligned Peripheral Metamers

TL;DR

The paper tackles rendering efficiency for wide-field immersive displays by introducing phase-aligned peripheral metamers that synthesize missing high-frequency content from the foveated base image using extrapolated local statistics. It develops a unified framework based on steerable quadrature filters to estimate local intensity, orientation, and phase, then synthesizes missing content via a Gabor-noise-like approach with cross-scale phase alignment and intensity control. Perceptual calibration and validation show that adding phase information substantially improves metamer fidelity, enabling approximately a 4x reduction in shading rate with minimal perceptual loss compared to full-resolution rendering. The work demonstrates perceptual and computational benefits over prior foveated-enhancement methods and provides a concrete method for generating efficient, perceptually faithful metamers in real time.

Abstract

Novel display technologies can deliver high-quality images across a wide field of view, creating immersive experiences. While rendering for such devices is expensive, most of the content falls into peripheral vision, where human perception differs from that in the fovea. Consequently, it is critical to understand and leverage the limitations of visual perception to enable efficient rendering. A standard approach is to exploit the reduced sensitivity to spatial details in the periphery by reducing rendering resolution, so-called foveated rendering. While this strategy avoids rendering part of the content altogether, an alternative promising direction is to replace accurate and expensive rendering with inexpensive synthesis of content that is perceptually indistinguishable from the ground-truth image. In this paper, we propose such a method for the efficient generation of an image signal that substitutes the rendering of high-frequency details. The method is grounded in findings from image statistics, which show that preserving appropriate local statistics is critical for perceived image quality. Based on this insight, we extrapolate several local image statistics from foveated content into higher spatial frequency ranges that are attenuated or omitted in the rendering process. This rich set of statistics is later used to synthesize a signal that is added to the initial rendering, boosting its perceived quality. We focus on phase information, demonstrating the importance of its alignment across space and frequencies. We calibrate and compare our method with state-of-the-art strategies, showing a significant reduction in the content that must be accurately rendered at a relatively small extra cost for synthesizing the additional signal.

Figures (11)

• Figure 1: The impact of recovering an increasing amount of local statistics: (a) solely local intensity is recovered (b) additionally, orientation is recovered (c) neighboring kernels are phase aligned.
• Figure 2: Extrapolation of parameters across multiple scales.
• Figure 3: Steerable quadrature basis filter set. The three $G$ and four $H$ filters form a steerable basis set each. If steered to an arbitrary rotation $\theta$, the resulting kernels $G_{\theta}$ and $H_{\theta}$ form a quadrature filter pair.
• Figure 4: Examples of phase, magnitude and resulting noise if we upscale the parameters across multiple scales. The number of reconstructed lines increases, as the intensity and phase field lose their locality.
• Figure 5: Exemplary frequency histogram of the Classroom scene after foveation and enhancement.