Optical aberrations in autonomous driving: Physics-informed parameterized temperature scaling for neural network uncertainty calibration

TL;DR

This work tackles uncertainty calibration under dataset shifts caused by windshield-induced optical aberrations in autonomous driving perception. It introduces Physics-Informed Parameterized Temperature Scaling (PIPTS), which injects a physical prior based on Zernike coefficients into a post-hoc calibration network to maintain reliable confidences for semantic segmentation. The study demonstrates that optical merit metrics like the Strehl ratio and Optical Informative Gain correlate more strongly with calibration performance than traditional half-Nyquist MTF metrics, and that PIPTS outperforms standard temperature scaling and vanilla PTS, with notable gains under mean and large aberrations. Practically, this approach enables more trustworthy uncertainty estimates, better system monitoring, and the ability to derive part-specific optical requirements for robust, safety-critical perception pipelines in autonomous driving.

Abstract

'A trustworthy representation of uncertainty is desirable and should be considered as a key feature of any machine learning method' (Huellermeier and Waegeman, 2021). This conclusion of Huellermeier et al. underpins the importance of calibrated uncertainties. Since AI-based algorithms are heavily impacted by dataset shifts, the automotive industry needs to safeguard its system against all possible contingencies. One important but often neglected dataset shift is caused by optical aberrations induced by the windshield. For the verification of the perception system performance, requirements on the AI performance need to be translated into optical metrics by a bijective mapping. Given this bijective mapping it is evident that the optical system characteristics add additional information about the magnitude of the dataset shift. As a consequence, we propose to incorporate a physical inductive bias into the neural network calibration architecture to enhance the robustness and the trustworthiness of the AI target application, which we demonstrate by using a semantic segmentation task as an example. By utilizing the Zernike coefficient vector of the optical system as a physical prior we can significantly reduce the mean expected calibration error in case of optical aberrations. As a result, we pave the way for a trustworthy uncertainty representation and for a holistic verification strategy of the perception chain.

Figures (9)

• Figure 1: The Chatterjee's rank correlation measure $\xi$ is compared to the Pearson correlation coefficient $\rho$ for the test function presented in Equation (\ref{['eq:test_function']}). The red curve indicates the functional relationship with Gaussian noise applied to it. Furthermore, subsamples are added randomly at the discontinuity ($x = \pm 2\pi$) to demonstrate the sensitivity of $\xi$ on the unexplained variance contribution, depicted by the blue dots.
• Figure 2: The Chatterjee's rank correlation measure $\xi$ is shown as a function of the relative cardinality of the subsample inserted at the location of the test function discontinuity at $x = \pm 2\pi$.
• Figure 3: The layout of the multi-task network for semantic segmentation and for predicting the effective Zernike coefficients of the optical system is shown. The multi-task network builds upon the UNET architecture with two coupled decoder heads and a downstream ResNet encoder for retrieving the Zernike coefficients of the second radial order. Additionally, the Fourier optical degradation model for the data augmentation process and the post-hoc PIPTS calibration network are indicated. The PIPTS calibrator extends the PTS approach by incorporating a physical inductive bias for ensuring the trustworthiness of the baseline multi-task network predictions under optical aberrations.
• Figure 4: Loss function study for the PIPTS calibration network. The loss is indicated for a random instance as a function of the calibration temperature in (a) with $\beta_{s} = 1000$ and $N_{b} = 10$. The smoothed ECE measure is plotted as a blue line and the corresponding gradient is visualized in (b). The discontinuity at the optimal temperature $\mathrm{T_{min}}$ indicates the need for an additional modulation function. The gradient of the total loss function, containing the modulation function and the temperature regularization term, is visualized in (d). It can be concluded that the total loss $\mathcal{L}$ is sufficiently continuous differentiable ($C^{1}$) for backpropagation. Furthermore, the gradient of the AUREC is plotted in (c) as a function of the calibration temperature. The number of peaks indicates, that the smoothing of the AUREC loss function by the softmax function was insufficient to ensure continuity. Hence, the AUREC loss function is inadequate for backpropagation and for neural network training respectively.
• Figure 5: The dependency of the mIoU (upper row) and the mECE (lower row) on the MTF at half Nyquist frequency (left column), the Strehl ratio (middle column) and the OIG (right column) is plotted. The Strehl ratio and the OIG demonstrate a superior correlation to the mIoU and the mECE in terms of the Chatterjee rank correlation measure than the MTF at half Nyquist frequency. As a consequence, the regression function from Equation (\ref{['eq:model_function']}) also fails to capture the non-existing relationship in the large-aberration regime but it performs well for the Strehl ratio and the OIG, which is quantitatively measured by the ratio of the Mean Squared Error (MSE) over the variance ($\sigma^2$), referred to as the unexplained variance component.