Benchmarking Burst Super-Resolution for Polarization Images: Noise Dataset and Analysis

TL;DR

The paper targets noise and ground-truth challenges in polarization imaging with snapshot polarization cameras. It introduces PolarNS to characterize polarization noise statistics and PolarBurstSR as a benchmark for polarization burst super-resolution, along with a physics-based Stokes-vector noise model that yields distributions for DoLP and AoLP. Through extensive validation, it demonstrates that polarization-specific training improves both intensity reconstruction and polarization accuracy compared to RGB-trained baselines, and it benchmarks multiple burst SR models on a polarization-focused setting. The work provides a public benchmark, including datasets, pretrained models, and pipelines, to advance denoising and high-resolution reconstruction in polarization imaging and to better understand noise propagation in polarization cues.$

Abstract

Snapshot polarization imaging calculates polarization states from linearly polarized subimages. To achieve this, a polarization camera employs a double Bayer-patterned sensor to capture both color and polarization. It demonstrates low light efficiency and low spatial resolution, resulting in increased noise and compromised polarization measurements. Although burst super-resolution effectively reduces noise and enhances spatial resolution, applying it to polarization imaging poses challenges due to the lack of tailored datasets and reliable ground truth noise statistics. To address these issues, we introduce PolarNS and PolarBurstSR, two innovative datasets developed specifically for polarization imaging. PolarNS provides characterization of polarization noise statistics, facilitating thorough analysis, while PolarBurstSR functions as a benchmark for burst super-resolution in polarization images. These datasets, collected under various real-world conditions, enable comprehensive evaluation. Additionally, we present a model for analyzing polarization noise to quantify noise propagation, tested on a large dataset captured in a darkroom environment. As part of our application, we compare the latest burst super-resolution models, highlighting the advantages of training tailored to polarization compared to RGB-based methods. This work establishes a benchmark for polarization burst super-resolution and offers critical insights into noise propagation, thereby enhancing polarization image reconstruction.

Figures (11)

• Figure 1: The top row displays the low-resolution burst input captured using a polarization camera. The second row presents close-up comparisons of different methods: the ground truth, the low-resolution input, the 2$\times$ super-resolution results of the RGB-trained BSRT method luo2022bsrt trained on a conventional RGB dataset, and the Polar-BSRT method trained on our polarization image dataset. Training with our dedicated polarization dataset significantly improves both the spatial resolution of the intensity image ($s_0$) and the angle of linear polarization (AoLP) map, demonstrating the importance of polarization-specific training for burst super-resolution.
• Figure 2: Our dataset examples presented in color intensity images $s_{0}$ (left) and their DoLP properties (right).
• Figure 3: (a) Description of the noise distribution: The noise distribution of a real vector $\mathbf{s}$ is illustrated in red, while the DoLP distribution for an arbitrary observation $\hat{\mathbf{s}}$ is depicted as the line integral of the blue circle. The AoLP distribution corresponds to the integral of the green line. (b) Distribution of observed DoLP in relation to true DoLP: The vertical solid line signifies the true DoLP, and the vertical dashed line indicates the biased maximum. (c) Distribution of observed DoLP concerning the ratio of $s_0$ and Stokes vector noise: The vertical solid line represents the true DoLP, while the vertical dashed line symbolizes the biased maximum. (d) Distribution of observed AoLP relative to the ratio of $s_{pol}$ and the Stokes vector noise.
• Figure 4: Histogram of polarization properties. The histogram values are normalized to a probability density scale. The black dashed line indicates the distribution predicted by our model. Each colored line represents the measured DoLP distribution at the corresponding ratio of the Stokes component value to the Stokes vector noise. (a) Histogram of DoLP. Each histogram displays the distribution for true DoLP values of 0.05, 0.1, and 0.2, respectively. The vertical line marks the location of the true DoLP value. (b) Histogram of AoLP.
• Figure 5: Histogram of the estimated DoLP bias, the DoLP standard deviation, and the AoLP standard deviation for the dataset. The histogram is computed in bins on a logarithmic scale, and the values are normalized to a probability density scale.