Multi-View Conformal Learning for Heterogeneous Sensor Fusion

TL;DR

This work tackles the need for uncertainty quantification in sensor fusion by introducing three multi-view conformal models (MV-A, MV-S, MV-I) that provide prediction sets with coverage guarantees. By treating each sensor as a separate view and leveraging conformal prediction, the authors demonstrate that multi-view approaches can reduce uncertainty (smaller prediction sets) and improve both traditional accuracy and conformal metrics compared to single-view baselines. MV-S, in particular, achieves the strongest performance across two heterogeneous datasets (HTAD and Berkeley MHAD), while MV-I offers a semi-conformal alternative via intersection of per-view sets. The findings have practical implications for safety-critical systems, where per-instance confidence is crucial for decision-making, and point to future work on heterogeneous models and sensor selection optimization.

Abstract

Being able to assess the confidence of individual predictions in machine learning models is crucial for decision making scenarios. Specially, in critical applications such as medical diagnosis, security, and unmanned vehicles, to name a few. In the last years, complex predictive models have had great success in solving hard tasks and new methods are being proposed every day. While the majority of new developments in machine learning models focus on improving the overall performance, less effort is put on assessing the trustworthiness of individual predictions, and even to a lesser extent, in the context of sensor fusion. To this end, we build and test multi-view and single-view conformal models for heterogeneous sensor fusion. Our models provide theoretical marginal confidence guarantees since they are based on the conformal prediction framework. We also propose a multi-view semi-conformal model based on sets intersection. Through comprehensive experimentation, we show that multi-view models perform better than single-view models not only in terms of accuracy-based performance metrics (as it has already been shown in several previous works) but also in conformal measures that provide uncertainty estimation. Our results also showed that multi-view models generate prediction sets with less uncertainty compared to single-view models.

Figures (23)

• Figure 1: non-conformal vs. conformal model. The conformal model produces prediction sets.
• Figure 2: Example of aggregating feature vectors from three views.
• Figure 3: The training data $\textbf{D'}$ is constructed by column binding the one-hot encoded label predictions of the first-level learners, the averaged scores, and the true labels y.
• Figure 4: Example of MV-I with three views (audio, images, and text).
• Figure 5: HTAD dataset with MV-S. Co-occurrence matrix (left), zero diagonal confusion matrix (right).