Mind the Exit Pupil Gap: Revisiting the Intrinsics of a Standard Plenoptic Camera

TL;DR

This work demonstrates that the exit pupil cannot be ignored when relating the camera-side light field to the scene in standard plenoptic cameras. It derives a formal, exit-pupil–aware connection between the object distance $o$ and the sub-aperture shift $S$, and provides explicit inverse relations and an alpha-based refocusing framework. The authors show how neglecting the exit pupil induces quantifiable errors in refocus and depth estimation, and they map how popular SPC calibration methods implicitly absorb these errors. Via a ray-tracing–based simulation platform and ten SPC configurations, they validate the theory, revise depth-calibration models (notably correcting Pertuz 2018's parametrization under Dansereau-style decoding), and publicly release data and a Blender-based camera generator to enable reproducibility. The results underscore the practical impact of exit-pupil considerations on calibration, refocusing accuracy, and depth reconstruction in plenoptic imaging, while outlining limitations and directions for extending to focused plenoptic cameras.

Abstract

Among the common applications of plenoptic cameras are depth reconstruction and post-shot refocusing. These require a calibration relating the camera-side light field to that of the scene. Numerous methods with this goal have been developed based on thin lens models for the plenoptic camera's main lens and microlenses. Our work addresses the often-overlooked role of the main lens exit pupil in these models and specifically in the decoding process of standard plenoptic camera (SPC) images. We formally deduce the connection between the refocusing distance and the resampling parameter for the decoded light field and provide an analysis of the errors that arise when the exit pupil is not considered. In addition, previous work is revisited with respect to the exit pupil's role and all theoretical results are validated through a ray-tracing-based simulation. With the public release of the evaluated SPC designs alongside our simulation and experimental data we aim to contribute to a more accurate and nuanced understanding of plenoptic camera optics.

Figures (16)

• Figure S1: Exemplary pipeline for SPC post-shot refocusing: A scene is captured by a virtual SPC shown without housing. The resulting raw image consists of a large number of microlens images and is subsequently decoded into a 4D light field representation which is visualized by a subset of the sub-aperture images dansereau2013decoding. By resampling the light field, a refocused image can be created ng2005lightfieldcamera. The correctly focused images were created based on parameters considering the exit pupil as described in \ref{['sec:SPC_Optics']} while the slightly defocused image are results from the directly calculated parameters without exit pupil consideration based on pertuz2018focus.
• Figure S2: Plenoptic camera modeled by a thick main lens combined with a thin lens MLA. The microlens pitch is described by $d_{\text{ML}}$ and the distance between neighboring microlens image centers (MICs) is denoted as $d_{\text{MLI}}$. Furthermore, $X$ describes the distance between the exit pupil and the camera-side principal plane and $d$ is the distance between $H_{\text{cam}}$ and the MLA. A complete notation overview is given in \ref{['append:notation']}.
• Figure S3: Integer (red) and metric (black) two plane parametrization of the light field. Here, $s_{px}$ describes the size of a sensor pixel and $f_m$ the focal length of a microlens which for an SPC coincides with the distance between MLA and sensor.
• Figure S4: Light field refocusing via shift of the virtual sensor, i.e. the $ST$-plane is moved to the image distance $\textbf{i}$. A ray $(s',u)$ can be associated with a ray $(s,u)$ by means of the triangle equality, i.e. $s=u+\frac{F}{F'}(s'-u)$.
• Figure S5: Left: Relative shift error based on $\lambda=\frac{\textbf{o}}{\textbf{o}_f}$. Mid/right: The two cases of relative object distance errors for the assumed camera focused at a finite distance which is met for a relative distance of $\frac{\textbf{o}}{\textbf{o}_f}=1$. Negative error values indicate an underestimation of the ground truth value while positive errors represent an overestimation.