Epistemic Uncertainty-aware Recommendation Systems via Bayesian Deep Ensemble Learning

TL;DR

The paper tackles epistemic uncertainty in recommender systems under data sparsity by introducing BDECF, a Bayesian deep ensemble framework that learns uncertainty-aware user/item representations, uses an attention-enhanced matching function, and ensembles ten weak models into a neural meta-learner. It combines Bayesian neural networks with deep ensembles to model weight uncertainty, and employs Bayes by Backprop with a diagonal Gaussian posterior to produce a predictive distribution and uncertainty estimates. The contributions include a novel representation-learning architecture with Bayesian last-layer embeddings, an attention-based scoring mechanism, and a trainable ensemble supermodel with principled data, parameter, and structural diversity, plus two uncertainty quantification schemes (reparameterization-based and ensemble-variance-based). Empirical results on MovieLens, Anime, and Book-Crossing datasets demonstrate robustness to sparsity and improved ranking metrics, with ablations confirming the value of the attention matching, ensemble, and uncertainty components, making the approach more reliable for real-world deployment.

Abstract

Recommending items to users has long been a fundamental task, and studies have tried to improve it ever since. Most well-known models commonly employ representation learning to map users and items into a unified embedding space for matching assessment. These approaches have primary limitations, especially when dealing with explicit feedback and sparse data contexts. Two primary limitations are their proneness to overfitting and failure to incorporate epistemic uncertainty in predictions. To address these problems, we propose a novel Bayesian Deep Ensemble Collaborative Filtering method named BDECF. To improve model generalization and quality, we utilize Bayesian Neural Networks, which incorporate uncertainty within their weight parameters. In addition, we introduce a new interpretable non-linear matching approach for the user and item embeddings, leveraging the advantages of the attention mechanism. Furthermore, we endorse the implementation of an ensemble-based supermodel to generate more robust and reliable predictions, resulting in a more complete model. Empirical evaluation through extensive experiments and ablation studies across a range of publicly accessible real-world datasets with differing sparsity characteristics confirms our proposed method's effectiveness and the importance of its components.

Figures (4)

• Figure 1: The process of representation learning. First, we learn the representation by our proposed network with weight uncertainty in the last layer. We multiply the embedding vectors element-wise, and after applying multi-head attention to this vector, we pass it through an MLP to get the final result.
• Figure 2: The architecture for the Supermodel.
• Figure 3: Quantifying uncertainty based on reparameterization trick
• Figure 4: Performance comparison with different dataset sizes