Physics Driven Image Simulation from Commercial Satellite Imagery

TL;DR

This paper presents a lidar-free, physics-driven pipeline to manufacture high-fidelity DIRSIG scenes for Earth regions using only commercial satellite imagery. By constructing a DSM from stereo satellite images, performing multi-stage material estimation with radiometric calibration and spectral matching, and modelling structures and terrain before procedurally populating the scene, the approach yields detailed, multi-spectral simulations spanning UV to LWIR. Key contributions include a DSM-based reconstruction workflow, a coarse-to-fine material estimation pipeline with a large spectral library, and mixture/ decal mapping plus automated DIRSIG file generation to enable scalable, high-detail synthetic data generation. The results demonstrate realistic UCSD-area simulations with substantial improvements in automation and detail, enabling algorithm development and testing across spectral bands with reduced manual effort. This work extends synthetic remote sensing capabilities to regions where lidar data is unavailable, offering practical impact for sensor design, data augmentation, and validation of image analysis pipelines.

Abstract

Physics driven image simulation allows for the modeling and creation of realistic imagery beyond what is afforded by typical rendering pipelines. We aim to automatically generate a physically realistic scene for simulation of a given region using satellite imagery to model the scene geometry, drive material estimates, and populate the scene with dynamic elements. We present automated techniques to utilize satellite imagery throughout the simulated scene to expedite scene construction and decrease manual overhead. Our technique does not use lidar, enabling simulations that could not be constructed previously. To develop a 3D scene, we model the various components of the real location, addressing the terrain, modelling man-made structures, and populating the scene with smaller elements such as vegetation and vehicles. To create the scene we begin with a Digital Surface Model, which serves as the basis for scene geometry, and allows us to reason about the real location in a common 3D frame of reference. These simulated scenes can provide increased fidelity with less manual intervention for novel locations on earth, and can facilitate algorithm development, and processing pipelines for imagery ranging from UV to LWIR $(200nm-20μm)$.

Figures (7)

• Figure 1: Simulated and real RGB imagery of a region of University of California, San Diego, showing the simulation closely matches a real WorldView-3 image.
• Figure 2: Fine-grained material matching for small region of the UCSD campus.
• Figure 3: A Google Earth screenshot of the simulation area. Approximately $2.75km^2$ of the UCSD campus in California.
• Figure 4: The colormapped and shaded DSM of the simulated region
• Figure 5: An illustration of some of the components that create the simulation. These include modeled buildings (blue), road network(black), populated trees (green), and the cars populated on the road network (yellow)