OpenViewer: Openness-Aware Multi-View Learning

TL;DR

OpenViewer addresses openness in multi-view learning by introducing a pseudo-unknown sample generation mechanism, an expression-enhanced deep unfolding network with four interpretable priors, and a perception-augmented open-set training regime. The framework decomposes multi-view integration into redundancy, consistency, diversity, and cross-view complementarity modules via ADMM, enabling transparent feature expression and robust handling of unknowns. Theoretical results establish interpretability bounds and convergence with rate $ ext{O}(1/T)$ within a radius $rac{ exteta \u03b5^{2}}{2}$ under fixed step sizes, ensuring stable generalization in open settings. Empirical results on six open multi-view datasets show OpenViewer outperforms baselines, with ablations confirming the contribution of each component to improved known-class recognition and unknown rejection.

Abstract

Multi-view learning methods leverage multiple data sources to enhance perception by mining correlations across views, typically relying on predefined categories. However, deploying these models in real-world scenarios presents two primary openness challenges. 1) Lack of Interpretability: The integration mechanisms of multi-view data in existing black-box models remain poorly explained; 2) Insufficient Generalization: Most models are not adapted to multi-view scenarios involving unknown categories. To address these challenges, we propose OpenViewer, an openness-aware multi-view learning framework with theoretical support. This framework begins with a Pseudo-Unknown Sample Generation Mechanism to efficiently simulate open multi-view environments and previously adapt to potential unknown samples. Subsequently, we introduce an Expression-Enhanced Deep Unfolding Network to intuitively promote interpretability by systematically constructing functional prior-mapping modules and effectively providing a more transparent integration mechanism for multi-view data. Additionally, we establish a Perception-Augmented Open-Set Training Regime to significantly enhance generalization by precisely boosting confidences for known categories and carefully suppressing inappropriate confidences for unknown ones. Experimental results demonstrate that OpenViewer effectively addresses openness challenges while ensuring recognition performance for both known and unknown samples. The code is released at https://github.com/dushide/OpenViewer.

Figures (14)

• Figure 1: Two multi-view environments and challenges.
• Figure 1: All compared methods’ t-SNE visualizations based on the representations $\mathbf{Z}$ of ESP-Game dataset.
• Figure 2: An overview of the proposed openness-aware multi-view learning framework (OpenViewer).
• Figure 2: All compared methods’ heatmap visualizations based on $\mathbf{Z}\mathbf{Z}^T$ of Animals dataset.
• Figure 3: Four multi-view priors and their relationships.

Theorems & Definitions (14)

• Theorem 1
• Theorem 2
• Definition 1
• Theorem 3
• proof
• Theorem 4
• proof
• Definition 2
• Theorem 5
• proof