Trusted Unified Feature-Neighborhood Dynamics for Multi-View Classification

TL;DR

This work tackles multi-view classification under domain gaps and conflicting views by introducing TUNED, a framework that fuses local feature-neighborhood structures within each view with global neighborhood consensus. A selective Markov random field (S-MRF) adaptively weighs cross-view evidence, while a shared parameterized evidence extractor learns Dirichlet-distributed evidence conditioned on local structures. Neighborhood-aware deep learning integrates local and global information to produce robust uncertainty estimates through Dirichlet parameters $\boldsymbol{\alpha}^v$ and corresponding beliefs $\boldsymbol{b}^v$ and uncertainty $u^v$, with losses that balance evidence quality and cross-view consistency. Empirical results on eight benchmarks show improved accuracy and resilience to conflicting views, outperforming DST-based fusion and other state-of-the-art evidential MVC methods; the approach advances reliable multi-view fusion in uncertain, heterogeneous data settings.

Abstract

Multi-view classification (MVC) faces inherent challenges due to domain gaps and inconsistencies across different views, often resulting in uncertainties during the fusion process. While Evidential Deep Learning (EDL) has been effective in addressing view uncertainty, existing methods predominantly rely on the Dempster-Shafer combination rule, which is sensitive to conflicting evidence and often neglects the critical role of neighborhood structures within multi-view data. To address these limitations, we propose a Trusted Unified Feature-NEighborhood Dynamics (TUNED) model for robust MVC. This method effectively integrates local and global feature-neighborhood (F-N) structures for robust decision-making. Specifically, we begin by extracting local F-N structures within each view. To further mitigate potential uncertainties and conflicts in multi-view fusion, we employ a selective Markov random field that adaptively manages cross-view neighborhood dependencies. Additionally, we employ a shared parameterized evidence extractor that learns global consensus conditioned on local F-N structures, thereby enhancing the global integration of multi-view features. Experiments on benchmark datasets show that our method improves accuracy and robustness over existing approaches, particularly in scenarios with high uncertainty and conflicting views. The code will be made available at https://github.com/JethroJames/TUNED.

Figures (11)

• Figure 1: Comparison of EDL-based methods. (a) Conventional models use DNNs $\{ f^v (\cdot) \}_{v=1}^V$ for evidence extraction, neglecting view-specific neighborhood structures and relying on the Dempster-Shafer theory (DST) for direct fusion, which can be unreliable with conflicting views. (b) Our method incorporates feature-neighborhood information via DNNs $\{ f^v (\cdot) \}_{v=1}^V$ and GNNs $\{ g^v (\cdot) \}_{v=1}^V$. The proposed Selective Markov Random Field (S-MRF) module dynamically fuses evidence, improving inference reliability without the need for hand-crafted loss functions.
• Figure 2: Illustration of the TUNED workflow, which comprises two stages. In the first stage, local view-specific feature-neighborhood (F-N) structures are extracted and fused to obtain evidence from the multi-view dataset. In the second stage, this evidence is integrated with global consensus F-N features through joint learning. The final step involves local-global evidence evaluation via EDL, followed by evidence fusion using the S-MRF module, leading to reliable inference outcomes.
• Figure 3: S-MRF weights for the HandWritten dataset. (a) Represents the weights under the normal setting, while (b) illustrates the weights under the conflict setting. And the red node indicates the view where conflict have been introduced.
• Figure 4: Robustness comparison: Impact of conflict views on model performance across different configurations.
• Figure 5: T-SNE visualization of Evidence. (a)-(c) show the t-SNE results for Evidence from three individual views. (d) illustrates the fused Evidence after t-SNE, highlighting the more cohesive data representation achieved through fusion.