Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges

TL;DR

Overall, this paper highlights the optical aberrations in automotive fisheye datasets, and the limitations of optical reality in simulated fisheye datasets, with a focus on computer vision in surround-view optical systems.

Abstract

In this paper, we provide a survey on automotive surround-view fisheye optics, with an emphasis on the impact of optical artifacts on computer vision tasks in autonomous driving and ADAS. The automotive industry has advanced in applying state-of-the-art computer vision to enhance road safety and provide automated driving functionality. When using camera systems on vehicles, there is a particular need for a wide field of view to capture the entire vehicle's surroundings, in areas such as low-speed maneuvering, automated parking, and cocoon sensing. However, one crucial challenge in surround-view cameras is the strong optical aberrations of the fisheye camera, which is an area that has received little attention in the literature. Additionally, a comprehensive dataset is needed for testing safety-critical scenarios in vehicle automation. The industry has turned to simulation as a cost-effective strategy for creating synthetic datasets with surround-view camera imagery. We examine different simulation methods (such as model-driven and data-driven simulations) and discuss the simulators' ability (or lack thereof) to model real-world optical performance. Overall, this paper highlights the optical aberrations in automotive fisheye datasets, and the limitations of optical reality in simulated fisheye datasets, with a focus on computer vision in surround-view optical systems.

Figures (19)

• Figure 1: Illustration of a surround-view camera system for automated driving. The cameras are FV (top left), MVR (top right), MVL (bottom left), and RV (bottom right) kumar2023surround.
• Figure 2: Illustration of 180° fisheye lens/camera combination. The captured image shows many optical artifacts associated with fisheye cameras, which we will discuss later in the paper. Note: the marked dotted boxes (orange) represent lateral chromatic aberration (red) and strong astigmatism or optical blur, geometric distortion effects on a building (blue), and both mechanical and optical vignetting at the periphery with slight shadow (green).
• Figure 3: Measurement of PSF for a simulated fisheye lens at the center and edge locationsburns2020application. Note that in the center, the PSF is approximately Gaussian, and has a relatively narrower peak indicating a sharper image. The Edge PSF is highly non-Gaussian and non-isotropic.
• Figure 4: PSF model of a full image wittpahl2018realisticlehmann2019resolution. Note the highly varying individual PSFs, from approximately isotropic in the center to highly non-isotropic at the periphery.
• Figure 5: Lateral chromatic aberration on a fisheye lens. Light splits into its components red, green, and blue (RGB) along the y-axis. We are only interested in the red, green, and blue components of the light, as an image sensor typically has only red, green, and blue pixels.