Parameter Blending for Multi-Camera Harmonization for Automotive Surround View Systems

TL;DR

This work addresses photometric seams in automotive surround-view caused by camera-specific AWB and GTM. It introduces a parameter blending framework that uses ISP metadata and a logistic blending curve to smoothly interpolate AWB and GTM between adjacent cameras before stitching, avoiding a fixed reference camera. The blending curve is defined as a modified logistic $f(x)=\frac{\text{SCALE}}{1+\exp(-x+6)}+\text{SHIFT}$ for $x\in[0,12]$, with $\text{SCALE}=1.005$, $\text{SHIFT}=-0.0025$. Experiments on four fisheye cameras show the metadata-based method reduces color and luminance differences and outperforms patch-based global color transfer in both visual quality and runtime, enabling real-time surround-view stitching.

Abstract

In a surround view system, the image color and tone captured by multiple cameras can be different due to cameras applying auto white balance (AWB), global tone mapping (GTM) individually for each camera. The color and brightness along stitched seam location may look discontinuous among multiple cameras which impacts overall stitched image visual quality. To improve the color transition between adjacent cameras in stitching algorithm, we propose harmonization algorithm which applies before stitching to adjust multiple cameras' color and tone so that stitched image has smoother color and tone transition between adjacent cameras. Our proposed harmonization algorithm consists of AWB harmonization and GTM harmonization leveraging Image Signal Processor (ISP)'s AWB and GTM metadata statistics. Experiment result shows that our proposed algorithm outperforms global color transfer method in both visual quality and computational cost.

Figures (6)

• Figure 1: Input to our harmonization algorithm are camera image and its metadata (i.e., auto white balance (AWB) and global tone mapping (GTM)). Our algorithm first conduct AWB harmonization to harmonize color and then followed by GTM harmonization to harmonize luminance. The harmonized images are then used for surround view stitching.
• Figure 2: Image1 is brighter along overlap boundary thus it has a decreased gain curve from middle column to boundary; Image 2 is darker along overlap boundary thus it has an increased gain curve from middle column to boundary.
• Figure 3: Left: original GTM look up table from metadata for two adjacent camera image. Image1 has brighter GTM curve than image2 before harmonization. Right: Harmonized GTM look up table at two image boundary. Image1's brightness is reduced while image2's brightness is increased after harmonization.
• Figure 4: Ablation on AWB and GTM Harmonization. AWB harmonization reduces greenish artifact on the left camera. GTM harmonization reduces luminance difference between right and rear camera.
• Figure 5: Ground projection and overlaps. Different cameras may not see the same objects in the overlap region.