Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Configurable Holography: Towards Display and Scene Adaptation

Yicheng Zhan, Liang Shi, Wojciech Matusik, Qi Sun, Kaan Akşit

TL;DR

This work tackles the challenge of retraining separate holography models for every display-scene configuration. It introduces a parameter-adaptive, RGB-only holography framework that conditions on $\lambda$, $Z$, $VD$, $d_x$, and $s$, complemented by a depth-estimation auxiliary task to improve 3D hologram fidelity. A teacher-student distillation pipeline yields a compact, fast student model that preserves image quality while delivering up to a 2x speed-up; depth-guided multitask learning further enhances hologram accuracy for RGB-only inputs. Evaluations on simulations and two holographic-display prototypes demonstrate continuous configurability across a broad parameter space and competitive performance relative to state-of-the-art methods, with practical implications for interactive holographic displays.

Abstract

Emerging learned holography approaches have enabled faster and high-quality hologram synthesis, setting a new milestone toward practical holographic displays. However, these learned models require training a dedicated model for each set of display-scene parameters. To address this shortcoming, our work introduces a highly configurable learned model structure, synthesizing 3D holograms interactively while supporting diverse display-scene parameters. Our family of models relying on this structure can be conditioned continuously for varying novel scene parameters, including input images, propagation distances, volume depths, peak brightnesses, and novel display parameters of pixel pitches and wavelengths. Uniquely, our findings unearth a correlation between depth estimation and hologram synthesis tasks in the learning domain, leading to a learned model that unlocks accurate 3D hologram generation from 2D images across varied display-scene parameters. We validate our models by synthesizing high-quality 3D holograms in simulations and also verify our findings with two different holographic display prototypes. Moreover, our family of models can synthesize holograms with a 2x speed-up compared to the state-of-the-art learned holography approaches in the literature.

Configurable Holography: Towards Display and Scene Adaptation

TL;DR

This work tackles the challenge of retraining separate holography models for every display-scene configuration. It introduces a parameter-adaptive, RGB-only holography framework that conditions on , , , , and , complemented by a depth-estimation auxiliary task to improve 3D hologram fidelity. A teacher-student distillation pipeline yields a compact, fast student model that preserves image quality while delivering up to a 2x speed-up; depth-guided multitask learning further enhances hologram accuracy for RGB-only inputs. Evaluations on simulations and two holographic-display prototypes demonstrate continuous configurability across a broad parameter space and competitive performance relative to state-of-the-art methods, with practical implications for interactive holographic displays.

Abstract

Emerging learned holography approaches have enabled faster and high-quality hologram synthesis, setting a new milestone toward practical holographic displays. However, these learned models require training a dedicated model for each set of display-scene parameters. To address this shortcoming, our work introduces a highly configurable learned model structure, synthesizing 3D holograms interactively while supporting diverse display-scene parameters. Our family of models relying on this structure can be conditioned continuously for varying novel scene parameters, including input images, propagation distances, volume depths, peak brightnesses, and novel display parameters of pixel pitches and wavelengths. Uniquely, our findings unearth a correlation between depth estimation and hologram synthesis tasks in the learning domain, leading to a learned model that unlocks accurate 3D hologram generation from 2D images across varied display-scene parameters. We validate our models by synthesizing high-quality 3D holograms in simulations and also verify our findings with two different holographic display prototypes. Moreover, our family of models can synthesize holograms with a 2x speed-up compared to the state-of-the-art learned holography approaches in the literature.
Paper Structure (20 sections, 4 equations, 7 figures, 3 tables)

This paper contains 20 sections, 4 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Learned Computer-Generated Holography.
  4. Multi-task Learning.
  5. Knowledge Distillation.
  6. Display-scene Parameter Adaptive Learned Holography
  7. Synthesizing 3D Holograms
  8. Learning To Synthesis Holograms
  9. Teacher Model.
  10. Phase-only hologram synthesis task.
  11. Light power estimation task.
  12. Depth estimation task.
  13. Student Model.
  14. Implementation
  15. Evaluation
  16. ...and 5 more sections

Figures (7)

  • Figure 1: Model overview. (Source: Lambo2014)
  • Figure 2: A collimated light illuminates a phase-only hologram, reconstructing the images at distance $Z$ with a certain depth $VD$.
  • Figure 3: Our teacher model. A lin2017feature is connected to every stage of our U-Net's decoder to leverage spatial information at varying scales from an input (RGB-only or RGB-D). woo2018cbam in our model introduce both channel and spatial attention mechanisms, while each decoder is conditioned with peak brightness, wavelength, volume depth, propagation distance, and pixel pitch. A layer he2015spatial enhances our model to aggregate spatial context for depth estimation. Our light head layer predicts the required light source powers. Here, $BCP$ represents the agarwal2022attention (RGB-only source: grapes2012).
  • Figure 4: Display-scene parameters and are used as the conditions of our model.
  • Figure 5: The overview of the student model.
  • ...and 2 more figures