A Renderer-Enabled Framework for Computing Parameter Estimation Lower Bounds in Plenoptic Imaging Systems

TL;DR

The paper develops a renderer-enabled framework to compute information-theoretic lower bounds on parameter estimation in plenoptic, passive NLOS imaging, leveraging the Hammersley-Chapman-Robbins bound and realistic forward models solved via ray tracing. It derives exact HCR bounds for Poisson and AWGN noise, introduces pixelwise Fisher information to localize information content, and demonstrates these methods on realistic hallway scenes, including analyses of rendering errors and their impact on the bounds. The work further extends to inexact rendering, providing a practical method to bound the true HCR via multi-N rendering and validating the bounds by comparing to maximum-likelihood estimates. Overall, the framework offers a robust benchmark for fundamental limits in plenoptic NLOS parameter estimation and yields insights into where information is most concentrated, guiding sensor placement and strategy. The findings show the bounds closely mirror ML performance in representative localization tasks, underscoring their practical relevance for designing and evaluating plenoptic imaging systems.

Abstract

This work focuses on assessing the information-theoretic limits of scene parameter estimation in plenoptic imaging systems. A general framework to compute lower bounds on the parameter estimation error from noisy plenoptic observations is presented, with a particular focus on passive indirect imaging problems, where the observations do not contain line-of-sight information about the parameter(s) of interest. Using computer graphics rendering software to synthesize the often-complicated dependence among parameter(s) of interest and observations, i.e. the forward model, the proposed framework evaluates the Hammersley-Chapman-Robbins bound to establish lower bounds on the variance of any unbiased estimator of the unknown parameters. The effects of inexact rendering of the true forward model on the computed lower bounds are also analyzed, both theoretically and via simulations. Experimental evaluations compare the computed lower bounds with the performance of the Maximum Likelihood Estimator on a canonical object localization problem, showing that the lower bounds computed via the framework proposed here are indicative of the true underlying fundamental limits in several nominally representative scenarios.

Figures (19)

• Figure 1: The rendering equation, explained graphically: (a) The proportion of incident light coming in from direction $\bm{\varphi}_i$ that gets reflected along direction $\bm{\varphi}_o$ is determined by the BRDF of the surface; (b) Light incident on a surface point $\mathbf{r}$, can be seen as light leaving from another point in the scene $g(\mathbf{r},\bm{\varphi}_i),\ \Rightarrow\mathbf{L}^{\rm in}_{\bm{\theta}^*}(\mathbf{r},-\bm{\varphi}_i) = \mathbf{L}_{\bm{\theta}^*}^{\rm out}(g(\mathbf{r},\bm{\varphi}_i),-\bm{\varphi}_i)$.
• Figure 2: A modified version of the Cornell Box scene with white walls and a single overhead light emitting white light uniformly across all wavelengths, rendered using $2^{12} = 4096$ samples per pixel (SPP). The walls and both spheres use the Mitsuba BSDF plugin roughplastic; the walls have $\alpha=0.1$, the green sphere has $\alpha=0.05$, and the red sphere has $\alpha = 0.4862$.
• Figure 3: Rendered gradients with $2^{12}=4096$ SPP ((a) - (c) and (g) - (i)) and $2^{14}=16384$ SPP ((d) - (f) and (j) - (l)), respectively. The top row shows gradients from differentiable rendering (AD) with respect to the red ball's: (a), (d) color, (b), (e) location, and (c), (f) radius. The bottom row shows the FD gradients with respect to the red ball's: (g), (j) color, (h), (k) location, and (i), (l) radius. The color gradients are relatively artifact-free, but the AD gradients for position and radius contains image artifacts in the form of groups spurious outlier pixels, which the FD gradients do not contain. The FD gradients were produced using central differencing from renders with $4096$ SPP each; results were averaged over 16 such rounds of renders. In this way, the FD gradients are the average of 16 gradient images, each from a different round of renders.
• Figure 4: Real hallway example scene: (a) A simplified hallway and room layout with dimensions marked, inspired by the real hallway image (b). The hallway in (a) is $4.9784\rm{m}$ long, the room has dimension $3.3274\rm{m}\times3.3782\rm{m}$, and both are $2.4384\rm{m}$ tall. The hallway is illuminated with 4 circular ceiling lights, the two closer of which have radius $4.445\rm{cm}$, and the two farther have radius $3.175\rm{cm}$. The room is illuminated with one circular ceiling light with radius $20.32\rm{cm}$, and a window to the outdoors which is $1.3716 \rm{m} \times 1.1557\rm{m}$. The camera $C_0$ is located $1.5108\rm{m}$ outside the hallway. The location and radius of a red spherical ball constitute the unknown scene parameter $\bm{\theta}^*$. Note that (a) and (c) include only the room at the end of the hallway on the right (no other rooms are modeled). Panel (c) depicts an image of the hallway scene of (a) rendered using Mitsuba 3jakob2022mitsuba3.
• Figure 5: Ball location manifold in the scene modeled after a real hallway. Ball locations along the manifold shown in black (approximately every $20$cm). The red marked ball locations show, respectively: (1) The start of the manifold (position of the ball closest to the sensor). (2) The last point of front-to-back displacement of the ball in the hallway, where the ball starts moving left-to-right. (3) The last point where the ball is fully in the sensors LOS. (4) The first point where the ball is fully outside of the sensor's LOS. (5) The left-to-right midpoint of the room (where the ball is located for radius HCR-LBs). (6) The last point in the manifold of ball locations.

Theorems & Definitions (10)

• Lemma 1: HCR Lower Bound
• Corollary 1.1: HCR Lower bound on the MSE
• Remark 1
• Remark 2
• Remark 3
• Remark 4
• Theorem 4.1: Effect of rendering error on $\lambda_P$
• Theorem 4.2: Effect of rendering error on $\lambda_G$
• Remark 5: Effect of rendering error on the overall HCR lower bound
• Claim 1: Relationship between $\widehat{\textup{HCR}}(\bm{\theta}^*_j)$ and $\textup{HCR}(\bm{\theta}^*_j)$