SynFog: A Photo-realistic Synthetic Fog Dataset based on End-to-end Imaging Simulation for Advancing Real-World Defogging in Autonomous Driving

TL;DR

SynFog addresses the realism gap in synthetic fog data by introducing a physically-based, end-to-end imaging pipeline that jointly models volumetric light transport in fog and a realistic camera chain. The dataset SynFog comprises 500 outdoor scenes under two lighting conditions and three fog densities, accompanied by depth, segmentation, and RAW data to support multi-modal training. Experiments show models trained on SynFog generalize better to real fog, improving visual defogging and object detection compared with ASM- or engine-based datasets, with fog-chamber validation corroborating realism. The work highlights the importance of accurate illumination, scattering, and sensor physics for robust defogging in autonomous driving and points to RAW-domain data and broader scattering media as future directions.

Abstract

To advance research in learning-based defogging algorithms, various synthetic fog datasets have been developed. However, existing datasets created using the Atmospheric Scattering Model (ASM) or real-time rendering engines often struggle to produce photo-realistic foggy images that accurately mimic the actual imaging process. This limitation hinders the effective generalization of models from synthetic to real data. In this paper, we introduce an end-to-end simulation pipeline designed to generate photo-realistic foggy images. This pipeline comprehensively considers the entire physically-based foggy scene imaging process, closely aligning with real-world image capture methods. Based on this pipeline, we present a new synthetic fog dataset named SynFog, which features both sky light and active lighting conditions, as well as three levels of fog density. Experimental results demonstrate that models trained on SynFog exhibit superior performance in visual perception and detection accuracy compared to others when applied to real-world foggy images.

Figures (10)

• Figure 1: The SynFog dataset comprises images under different lighting, with the first row captured in natural sky light and the second with extra light sources like street lamps and automotive lighting. This dataset covers three levels of fog density across each scene, corresponding visibility of 600 m, 300 m, 150 m. Additionally, pixel-accurate depth data and segmentation labels for each scene are also provided.
• Figure 2: Relationship between noise level and fog density in real-world foggy images. Using regional contrast stretching for image enhancement, it can be observed that the noise level increases with the concentration of fog.
• Figure 3: Foggy scene simulation methods.
• Figure 4: End-to-end foggy image simulation pipeline. The spectral radiance data is rendered using volumetric path tracing and passing through a realistic optics model before reaching the sensor plane. Subsequently, the irradiance is converted into a digital image through the sensor model. The raw image from the sensor is then processed by an ISP module to generate the final foggy image.
• Figure 5: Noise flow and sensor simulation.