Propagating Similarity, Mitigating Uncertainty: Similarity Propagation-enhanced Uncertainty for Multimodal Recommendation

TL;DR

This paper tackles modality-specific noise in multimodal recommendation by introducing SPUMR, which propagates similarity through Modality and Collaborative Similarity Graphs to refine content- and behavior-based representations. It then applies uncertainty-aware aggregation, modeling each modality-entity representation as a Gaussian and fusing them via a gating mechanism that downweights high-uncertainty experts. The approach combines KL-regularized stochastic embeddings with a Bayesian and contrastive objective, achieving statistically significant improvements over state-of-the-art baselines on three Amazon datasets. The results demonstrate improved robustness to noisy modal features and better utilization of similarity signals for both users and items, with ablations and hyper-parameter analyses supporting the effectiveness of each component. The work has practical implications for robust, noise-aware multimodal recommendations and suggests avenues for integrating uncertainty-aware fusion with larger multimodal models.

Abstract

Multimodal Recommendation (MMR) systems are crucial for modern platforms but are often hampered by inherent noise and uncertainty in modal features, such as blurry images, diverse visual appearances, or ambiguous text. Existing methods often overlook this modality-specific uncertainty, leading to ineffective feature fusion. Furthermore, they fail to leverage rich similarity patterns among users and items to refine representations and their corresponding uncertainty estimates. To address these challenges, we propose a novel framework, Similarity Propagation-enhanced Uncertainty for Multimodal Recommendation (SPUMR). SPUMR explicitly models and mitigates uncertainty by first constructing the Modality Similarity Graph and the Collaborative Similarity Graph to refine representations from both content and behavioral perspectives. The Uncertainty-aware Preference Aggregation module then adaptively fuses the refined multimodal features, assigning greater weight to more reliable modalities. Extensive experiments on three benchmark datasets demonstrate that SPUMR achieves significant improvements over existing leading methods.

Figures (4)

• Figure 1: Illustration of the modal-specific uncertainty. The left image shows that text descriptions for products can be unclear and based on opinion, while the right shows that the same watch category can look very different (percentage represents visual similarity).
• Figure 2: The overview structure of SPUMR.
• Figure 3: Performance of SPUMR under different values of $\lambda_{KL}$ and $\lambda_{U}$ on three datasets.
• Figure 4: The distribution of the user representations learned by MMGCN, FREEDOM, and SPUMR on the Baby dataset.