COHESION: Composite Graph Convolutional Network with Dual-Stage Fusion for Multimodal Recommendation

TL;DR

COHESION tackles multimodal recommendation by unifying fusion and representation learning. It introduces a dual-stage fusion mechanism that refines all modalities via the behavior modality and then fuses them at a late stage, complemented by a composite graph convolutional network that leverages both heterogeneous (user-item, user-user, item-item) and homogeneous graphs. An adaptive optimization objective balances modality contributions to improve robustness when multiple modalities are available. Extensive experiments on three Amazon datasets show COHESION consistently outperforms strong baselines and demonstrates robustness to data sparsity, efficiency, and stability across hyper-parameter settings.

Abstract

Recent works in multimodal recommendations, which leverage diverse modal information to address data sparsity and enhance recommendation accuracy, have garnered considerable interest. Two key processes in multimodal recommendations are modality fusion and representation learning. Previous approaches in modality fusion often employ simplistic attentive or pre-defined strategies at early or late stages, failing to effectively handle irrelevant information among modalities. In representation learning, prior research has constructed heterogeneous and homogeneous graph structures encapsulating user-item, user-user, and item-item relationships to better capture user interests and item profiles. Modality fusion and representation learning were considered as two independent processes in previous work. In this paper, we reveal that these two processes are complementary and can support each other. Specifically, powerful representation learning enhances modality fusion, while effective fusion improves representation quality. Stemming from these two processes, we introduce a COmposite grapH convolutional nEtwork with dual-stage fuSION for the multimodal recommendation, named COHESION. Specifically, it introduces a dual-stage fusion strategy to reduce the impact of irrelevant information, refining all modalities using ID embedding in the early stage and fusing their representations at the late stage. It also proposes a composite graph convolutional network that utilizes user-item, user-user, and item-item graphs to extract heterogeneous and homogeneous latent relationships within users and items. Besides, it introduces a novel adaptive optimization to ensure balanced and reasonable representations across modalities. Extensive experiments on three widely used datasets demonstrate the significant superiority of COHESION over various competitive baselines.

Figures (7)

• Figure 1: The overview structure of COHESION.
• Figure 2: Effect of early fusion.
• Figure 3: Visualization of distribution through t-SNE. We randomly sample 500 items from Baby dataset. Red, green, and blue represent the ID, textual, and visual modalities.
• Figure 4: Sparsity degree analysis on three datasets.
• Figure 5: Convergence study in terms of Recall@20.