Monte Carlo Path Tracing and Statistical Event Detection for Event Camera Simulation

TL;DR

This work tackles the challenge of realistically simulating event cameras with physically based Monte Carlo rendering by introducing an adaptive sampler that operates on logarithmic luminance. By modeling $\ln L$ as normal via the central limit theorem and performing a one-tailed $t$-test against a threshold $\theta$, the renderer terminates sampling early at pixels unlikely to produce events, achieving substantial speedups while preserving event fidelity. Implemented on PBRT-v4, the approach demonstrates significant performance gains (often $>6\times$) across multiple scenes, with quantitative metrics indicating high temporal precision in event timing. The study provides a practical framework for high-fidelity event-camera simulation and lays groundwork for broader CV applications requiring realistic event data.

Abstract

This paper presents a novel event camera simulation system fully based on physically based Monte Carlo path tracing with adaptive path sampling. The adaptive sampling performed in the proposed method is based on a statistical technique, hypothesis testing for the hypothesis whether the difference of logarithmic luminances at two distant periods is significantly larger than a predefined event threshold. To this end, our rendering system collects logarithmic luminances rather than raw luminance in contrast to the conventional rendering system imitating conventional RGB cameras. Then, based on the central limit theorem, we reasonably assume that the distribution of the population mean of logarithmic luminance can be modeled as a normal distribution, allowing us to model the distribution of the difference of logarithmic luminance as a normal distribution. Then, using Student's t-test, we can test the hypothesis and determine whether to discard the null hypothesis for event non-occurrence. When we sample a sufficiently large number of path samples to satisfy the central limit theorem and obtain a clean set of events, our method achieves significant speed up compared to a simple approach of sampling paths uniformly at every pixel. To our knowledge, we are the first to simulate the behavior of event cameras in a physically accurate manner using an adaptive sampling technique in Monte Carlo path tracing, and we believe this study will contribute to the development of computer vision applications using event cameras.

Figures (6)

• Figure 1: The result of a preliminary test of the v2e system hu2021v2e using the video frames obtained by Monte Carlo path tracing with 256 samples per pixel. Due to the lack of path samples, video frames input to the v2e system and the resulting event video are significantly noisy.
• Figure 2: Illustration representing the difference between one-tailed and two-tailed hypothesis testing. While two-tailed testing discards the null hypothesis in both cases where an event occurs or no event occurs, one-tailed testing discards the hypothesis only in the case where no event occurs.
• Figure 3: Comparison of event rendering results obtained by mean-only, two-tailed, and one-tailed approaches. While the result of the two-tailed approach includes significantly much noise, the mean-only and one-tailed approaches obtain equivalent results.
• Figure 4: Rendering results of the proposed system and the sampling ratio compared to the uniform sampling. The color bar applied to the sample maps is the same as that in \ref{['fig:teaser']}.
• Figure 9: A detailed analysis of the difference in event occurrence between the baseline and our method. The change in $\ln L$ at pixel $(83, 70)$, which exhibits the maximum event deviation error in the "living room" scene, is shown with the time points of event occurrence, where positive and negative events are marked by "$\blacktriangle$" and "$\blacktriangledown$" symbols, respectively. Best viewed on screen.