Gaussians on Fire: High-Frequency Reconstruction of Flames

TL;DR

This work tackles high-frequency flame reconstruction from only three synchronized cameras by decoupling static background and dynamic fire, then initializing dynamic Gaussians from a fused 3D flow and evolving them with per-Gaussian lifetimes and velocities via a FreeTimeGS representation. It integrates dense stereo depth, monocular priors, and flow-based motion to achieve temporally coherent, photorealistic renderings under sparse-view conditions, while explicitly accounting for rolling shutter and hardware synchronization. The authors validate on a real-world, hardware-synchronized three-camera dataset with diverse fuels, showing strong flame-focused metrics and depth plausibility against 4D Gaussian Splatting baselines. This approach enables practical high-frequency fire reconstruction on commodity hardware, with potential for downstream tasks like motion analysis and robotics navigation in emissive environments.

Abstract

We propose a method to reconstruct dynamic fire in 3D from a limited set of camera views with a Gaussian-based spatiotemporal representation. Capturing and reconstructing fire and its dynamics is highly challenging due to its volatile nature, transparent quality, and multitude of high-frequency features. Despite these challenges, we aim to reconstruct fire from only three views, which consequently requires solving for under-constrained geometry. We solve this by separating the static background from the dynamic fire region by combining dense multi-view stereo images with monocular depth priors. The fire is initialized as a 3D flow field, obtained by fusing per-view dense optical flow projections. To capture the high frequency features of fire, each 3D Gaussian encodes a lifetime and linear velocity to match the dense optical flow. To ensure sub-frame temporal alignment across cameras we employ a custom hardware synchronization pattern -- allowing us to reconstruct fire with affordable commodity hardware. Our quantitative and qualitative validations across numerous reconstruction experiments demonstrate robust performance for diverse and challenging real fire scenarios.

Figures (8)

• Figure 1: Overview of our pipeline: Given the video of a fire from three views, we first calibrate and register cameras using COLMAP and estimate dense stereo depth maps via PatchMatchStereo, complemented by aligned monocular depth predictions for regularization. The static background scene is optimized with vanilla 3D Gaussian Splatting using this fused depth initialization. Dynamic regions such as fire are reconstructed by projecting dense optical flows into a 3D voxel grid to estimate a volumetric motion field, which initializes FreeTimeGS-based dynamic Gaussians.
• Figure 2: Fire removal for dynamic scenes: we stack all frames along the sequence dimension and compute the minimum intensity projection over this axis to obtain an image of the static scene with most of the fire removed. We show a scene with a small (a,b) and larger flame (c,d) before and after this process.
• Figure 3: Intermediate data used for scene reconstruction: (a) Input RGB frame, (b) monocular depth prediction, (c) depth from structure-from-motion (SfM) (d) segmentation mask of the dynamic region, (e) estimated 2D optical flow, and (f) fused voxelized 3D flow field used to initialize dynamic Gaussians.
• Figure 4: Multi-view capture of a fire: we capture the RGB images from the three camera viewpoints (a)-(c) and show the corresponding rendered depth maps (d)-(f). The depth reconstructions demonstrate consistent geometry across views despite strong appearance variations caused by the dynamic flames.
• Figure 5: Illustration of rolling-shutter effects. (a) Image rendered with a global-shutter model, (b) image with rolling-shutter readout showing temporal distortion for fast motion, and (c) grayscale difference highlighting the resulting spatiotemporal displacement.