Digital Methods to Quantify Sensor Output Uncertainty in Real Time

Abstract

Modern data-driven applications that make real-time decisions increasingly depend on advanced sensors which use pre-stored calibration data. In such applications, accurate characterization of sensor output uncertainty is important for reliable data interpretation. Here, we present a method for real-time on-device dynamic uncertainty quantification for sensor outputs which depend on pre-stored calibration data. We show how sensor calibration compensation equations (essential in advanced sensing systems) propagate uncertainties resulting from the quantization of calibration parameters to the sensor output. We use a low-cost thermal sensor as a motivating example and show these ideas are practical and possible on actual embedded sensor systems by prototyping them on two commercially-available uncertainty tracking hardware platforms. One has average power dissipation 16.7 mW and achieves 42.9x speedup compared to the equal-accuracy Monte Carlo computation (the status quo), and the other 147.15 mW and achieves 94.4x speedup. We present a proof-of-usefulness application using the quantified uncertainty in edge detection over ten test scenes where we show accuracy and precision average improvement by 4.97 and 40.25 percentage points, respectively, trading off sensitivity. Another application example examines uncertainty quantification for four different calibration-data storage scenarios and compute that a 48% increase in memory yields 75% smaller uncertainty metrics over the baseline.

Figures (12)

• Figure 1: Simplified illustrative diagram for a calibration-compensated sensor. (A) The environment produces a physical phenomenon as an analog, potentially non-electrical, signal related to a quantity (B) we want to measure. (C) On the sensor device, a component picks up the signal and transduces it to an electrical signal (D). (E) An amplifier amplifies the electrical signal and analog circuits process it further. (F) An analog-to-digital converter turns measurements into digital values (G). These values do not have an obvious correspondence to the measurand. (H) The calibration phase uses reference values to fit a function (conversion routine) which applications (J) will use to compute measurand values (K) from the raw readings (G). The conversion routine uses parameters from the calibration phase (J) that the manufacturer pre-stored in the sensor device memory (L). Because these parameters are constrained in precision by finite digital memories, quantization errors (M) in the extracted parameters cause the sensor output of the conversion routines to contain significant amounts of epistemic uncertainty (N). (O) Conventional approaches can quantify this uncertainty in a static off-device offline approach: this work presents a method for dynamic on-device real-time uncertainty quantification for sensor outputs which depend on pre-stored calibration data.
• Figure 2: a, Commercially-available system-on-module UxHw-FPGA-5k for native uncertainty tracking with 12 clock speed, 128 RAM, 12 base power, 10$\times$15 size. b, Commercially-available system-on-module UxHw-FPGA-17k for native uncertainty tracking with 45 clock speed, 320 RAM, 99 base power, 10$\times$15 size. c, Conversion routine output distribution when running on Ux-FPGA-5k and Ux-FPGA-17k with native uncertainty tracking. This result approximates the corresponding output distribution from the Monte Carlo execution with a Wasserstein distance of 0.0189℃, in under 5 per pixel and with average power 16.7 on UxHw-FPGA-5k. By contrast, to ensure same level of accuracy with 90% confidence using Monte Carlo execution, the embedded system must re-execute the conversion routines at least 3000 times, which takes 203.
• Figure 3: A, Experimental measurement setup: an MLX90640 melexis2019mlx thermal sensor, controlled by a microcontroller, measures infrared radiation emanating from a FLUKE 4180 infrared calibration source. A FLUKE 289 thermocouple monitors ambient temperature. Red: key items. Green: supporting items, out of frame. Blue: sensor field-of-view projection. B, The thermal image sensor outut of the setup in \ref{['fig:MlxSensorExperimentalSetupTopView']}.A, after conversion. C, Distribution of due-to-limited-precision uncertainty for three selected pixels of Figure \ref{['fig:MlxSensorExperimentalSetupScene']}. Our method uncovers the dynamic uncertainty distribution of estimated temperature for every pixel of the image.
• Figure 4: a, The standard deviation of conversion routine output temperature for all 768 pixels of a single measurement of a single MLX90640 sensor instance. For a target temperature of 100℃, the output uncertainty has average standard deviation 0.94℃ for the center pixels, while for the corner pixels std average 1.5℃ and max 1.75℃. Overall, the standard deviation has an increasing trend as pixels approach the frame edge of the sensor output and as the temperature increases. b, Summary for the probable absolute and relative errors because of representation uncertainty of the MLX90640 calibration data (conventional sensor output vs distribution), from all 768 pixels of four tested sensor instances, each tested for 21 target temperatures (322560 sensor output distributions). MAE: Mean Absolute Error, MaxAE: Max Absolute Error, StDev: Standard Deviation, 95% CIs: Size of 95% Confidence Intervals, MRE: Mean Relative Error, MaxRE: Max Relative Error.
• Figure 5: The uncertainty information helps against false positives in edge detection.A, The conventional sensor output. B, Canny edge detector. C, The result of the Canny edge detector for the conventional sensor output yields false positive edges close to the bottom corners of the frame. D, Small horizontal fluctuations leads to false positive edges. E, The distribution of the temperature frame samples from Section \ref{['section:conversionCalibrationRoutines']}. F, The result of the Canny execution using the temperature frame distribution yields probabilistic edges, with the highest probabilities for the true positive edges. False-positive edges in every sample frame: 26.8±8. G, The true edges have edge probability equal to one. We filter the data with probability lower than one and the true positive edges remain.