Event-based Background-Oriented Schlieren

TL;DR

This paper introduces event-based background-oriented schlieren (BOS), a theory-grounded approach that uses event cameras to estimate the temporal density derivative $\frac{\partial \rho}{\partial t}$ in schlieren scenes. By extending the linearized event generation model and parameterizing the flow via $\mathbf{q}=\frac{\partial \rho}{\partial t}$, the method recovers schlieren-induced density changes through a variational objective combining event data with a frame reference. The authors provide a multi-scale optimization framework, a physically motivated regularization, and a frame-event fusion pipeline that yields accurate optical-flow-like estimates, robust HDR performance, and high-temporal-resolution visualization (kymograms). They validate the method on co-registered frame/event data, demonstrate improved performance over baselines, and release a public dataset and code to foster further exploration of event-based BOS in fluid dynamics and imaging.

Abstract

Schlieren imaging is an optical technique to observe the flow of transparent media, such as air or water, without any particle seeding. However, conventional frame-based techniques require both high spatial and temporal resolution cameras, which impose bright illumination and expensive computation limitations. Event cameras offer potential advantages (high dynamic range, high temporal resolution, and data efficiency) to overcome such limitations due to their bio-inspired sensing principle. This paper presents a novel technique for perceiving air convection using events and frames by providing the first theoretical analysis that connects event data and schlieren. We formulate the problem as a variational optimization one combining the linearized event generation model with a physically-motivated parameterization that estimates the temporal derivative of the air density. The experiments with accurately aligned frame- and event camera data reveal that the proposed method enables event cameras to obtain on par results with existing frame-based optical flow techniques. Moreover, the proposed method works under dark conditions where frame-based schlieren fails, and also enables slow-motion analysis by leveraging the event camera's advantages. Our work pioneers and opens a new stack of event camera applications, as we publish the source code as well as the first schlieren dataset with high-quality frame and event data. https://github.com/tub-rip/event_based_bos

Figures (16)

• Figure 1: In background-oriented schlieren (BOS) imaging local density gradient variations between a camera and a background pattern lead to tiny perceived changes on the image plane. We present how to combine events and frames to observe schlieren in the scene and how to leverage the advantages of event cameras to visualize gas streams such as the human breath.
• Figure 2: Background-Oriented Schlieren (BOS) setup.
• Figure 3: Frame-based BOS and event-based BOS.
• Figure 4: (a) Actual synchronized data recording system, combining an event camera and a frame-based camera via a beamsplitter (\ref{['sec:recordingSetup']}). (b) Data: events (red and blue, colored according to polarity) during a short time window overlaid on a grayscale frame (a $100\times 100$ pixel region for better visualization).
• Figure 5: Block diagram of the objective $E_\text{data}$ in \ref{['eq:Edata']}. On the top branch, events are integrated in time using \ref{['eq:brightnessIncrementEvents']} and smoothed with a Gaussian kernel ($\sigma=2$ px) to produce the measured brightness increment image $\Delta L$. The bottom branch shows how to compute the predicted brightness increment $\Delta \hat{L}$ from the frame and the problem unknowns: the translation field $\mathbf{p}$ and the Poisson parameters of the flow, $\mathbf{q}$. The flow $\mathbf{v}$ and $\mathbf{p}$ are pseudo-colored (see color wheel). Same data as \ref{['fig:cocapture']}.