Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Inspired Multi Modal Recommendation

Jie Yang, Chenyang Gu, Zixuan Liu

TL;DR

This work tackles modal confounding and interaction bias in multimodal recommender systems by introducing a causal-inspired framework (CaMRec). It combines a dual-channel diffusion module to identify latent modal confounders, back-door adjustment with a learnable environment codebook and hierarchical matching, and front-door adjustment with causal topology reconstruction to extract true user–item causal signals. The approach yields a deconfounded causal subgraph and interpretable environment clusters, achieving consistent improvements over state-of-the-art baselines on three real-world datasets. The findings demonstrate that causal interventions can substantially enhance accuracy and robustness in multimodal recommendations, with clear mechanisms for interpretability.

Abstract

Multimodal recommender systems enhance personalized recommendations in e-commerce and online advertising by integrating visual, textual, and user-item interaction data. However, existing methods often overlook two critical biases: (i) modal confounding, where latent factors (e.g., brand style or product category) simultaneously drive multiple modalities and influence user preference, leading to spurious feature-preference associations; (ii) interaction bias, where genuine user preferences are mixed with noise from exposure effects and accidental clicks. To address these challenges, we propose a Causal-inspired multimodal Recommendation framework. Specifically, we introduce a dual-channel cross-modal diffusion module to identify hidden modal confounders, utilize back-door adjustment with hierarchical matching and vector-quantized codebooks to block confounding paths, and apply front-door adjustment combined with causal topology reconstruction to build a deconfounded causal subgraph. Extensive experiments on three real-world e-commerce datasets demonstrate that our method significantly outperforms state-of-the-art baselines while maintaining strong interpretability.

Causal Inspired Multi Modal Recommendation

TL;DR

This work tackles modal confounding and interaction bias in multimodal recommender systems by introducing a causal-inspired framework (CaMRec). It combines a dual-channel diffusion module to identify latent modal confounders, back-door adjustment with a learnable environment codebook and hierarchical matching, and front-door adjustment with causal topology reconstruction to extract true user–item causal signals. The approach yields a deconfounded causal subgraph and interpretable environment clusters, achieving consistent improvements over state-of-the-art baselines on three real-world datasets. The findings demonstrate that causal interventions can substantially enhance accuracy and robustness in multimodal recommendations, with clear mechanisms for interpretability.

Abstract

Multimodal recommender systems enhance personalized recommendations in e-commerce and online advertising by integrating visual, textual, and user-item interaction data. However, existing methods often overlook two critical biases: (i) modal confounding, where latent factors (e.g., brand style or product category) simultaneously drive multiple modalities and influence user preference, leading to spurious feature-preference associations; (ii) interaction bias, where genuine user preferences are mixed with noise from exposure effects and accidental clicks. To address these challenges, we propose a Causal-inspired multimodal Recommendation framework. Specifically, we introduce a dual-channel cross-modal diffusion module to identify hidden modal confounders, utilize back-door adjustment with hierarchical matching and vector-quantized codebooks to block confounding paths, and apply front-door adjustment combined with causal topology reconstruction to build a deconfounded causal subgraph. Extensive experiments on three real-world e-commerce datasets demonstrate that our method significantly outperforms state-of-the-art baselines while maintaining strong interpretability.
Paper Structure (17 sections, 18 equations, 2 figures, 2 tables)

This paper contains 17 sections, 18 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Causal Interpretation
  4. Problem Statement
  5. A Causal Interpretation for Multimodal Recommendation
  6. Confounders and Causal Treatments
  7. Front-Door Adjustment For $G$
  8. Model Instantiation
  9. Back-door Adjustment
  10. Front-door Adjustment
  11. Dual Channel Cross-Modal Diffusion Module
  12. Back-Door and Front-Door Adjustment
  13. Back-Door Adjustment.
  14. Front-Door Adjustment.
  15. Experiment
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Comparison of multimodal feature extraction approaches for product descriptions. The top path illustrates a straightforward naive fusion of text and image features, leading to potential misinterpretation due to unaddressed modal confounders. The bottom path shows an enhanced approach using dual-channel diffusion and back-door adjustment to clean and integrate features, resulting in a more accurate representation of the product’s unique qualities, reducing biases from modal confounders.
  • Figure 2: Overview of the causal-inspired multimodal recommendation framework. The model uses a cross-modal conditional generator to process textual and visual features, incorporating a dual-channel diffusion mechanism to estimate common factors for each modality. The system employs back-door adjustment to eliminate modal confounders, followed by a stratified intervention. Topology reconstruction, powered by InfoNCE loss, ensures that the learned causal relationships between modalities and user preferences remain robust and unbiased.