GaVaMoE: Gaussian-Variational Gated Mixture of Experts for Explainable Recommendation

TL;DR

GaVaMoE tackles the challenge of explainable recommendations under sparse user-item interactions by integrating a VAE-GMM module for learning latent preferences and clustering users with similar behaviors, with a cluster-aware multi-gating Mixture of Experts to generate personalized explanations. The two-stage training jointly optimizes collaborative preference understanding and explanation quality, using an ELBO-based objective followed by explanation-focused optimization. Empirical results on three real-world datasets show GaVaMoE outperforms baselines in explanation quality, personalization, and consistency, with strong robustness in sparse data scenarios due to cross-cluster knowledge transfer. The approach offers a scalable pathway to produce fluent, tailored explanations in practical settings, improving user trust and engagement in recommender systems.

Abstract

Large language model-based explainable recommendation (LLM-based ER) systems show promise in generating human-like explanations for recommendations. However, they face challenges in modeling user-item collaborative preferences, personalizing explanations, and handling sparse user-item interactions. To address these issues, we propose GaVaMoE, a novel Gaussian-Variational Gated Mixture of Experts framework for explainable recommendation. GaVaMoE introduces two key components: (1) a rating reconstruction module that employs Variational Autoencoder (VAE) with a Gaussian Mixture Model (GMM) to capture complex user-item collaborative preferences, serving as a pre-trained multi-gating mechanism; and (2) a set of fine-grained expert models coupled with the multi-gating mechanism for generating highly personalized explanations. The VAE component models latent factors in user-item interactions, while the GMM clusters users with similar behaviors. Each cluster corresponds to a gate in the multi-gating mechanism, routing user-item pairs to appropriate expert models. This architecture enables GaVaMoE to generate tailored explanations for specific user types and preferences, mitigating data sparsity by leveraging user similarities. Extensive experiments on three real-world datasets demonstrate that GaVaMoE significantly outperforms existing methods in explanation quality, personalization, and consistency. Notably, GaVaMoE exhibits robust performance in scenarios with sparse user-item interactions, maintaining high-quality explanations even for users with limited historical data.

Figures (6)

• Figure 1: Comparison between traditional LLM-based recommendation systems and GaVaMoE. (a) Traditional approaches directly map user-item IDs to LLM prompts for explanation generation. (b) GaVaMoE first encodes user-item interactions through VAE-GMM clustering, then routes them to specialized experts via multi-gating for personalized explanations.
• Figure 2: Architecture overview of GaVaMoE. The framework consists of N stacked blocks, each containing: (1) a VAE-GMM module that learns collaborative preferences and clusters users with similar behaviors, and (2) a multi-gating mixture of experts that routes user-item pairs to specialized experts based on learned clusters. The model employs a two-stage training process, first optimizing collaborative preference learning and clustering, then fine-tuning for personalized explanation generation.
• Figure 3: Ablation study results comparing GaVaMoE variants across TripAdvisor, Amazon, and Yelp datasets.
• Figure 4: Impact of block and expert configurations on GaVaMoE performance. Left: Bar chart showing different combinations of transformer blocks (blue) and experts (green) tested in the experiments. Right: Line plot demonstrating the corresponding BLEU-4 and BERTScore metrics for each configuration, where configurations are denoted as "blocks-experts" (e.g., "32-12" represents 32 blocks with 12 experts).
• Figure 5: Visualization of latent space distributions in GaVaMoE for different numbers of user clusters.