Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are Representation Disentanglement and Interpretability Linked in Recommendation Models? A Critical Review and Reproducibility Study

Ervin Dervishaj, Tuukka Ruotsalo, Maria Maistro, Christina Lioma

TL;DR

The paper investigates whether representation disentanglement and interpretability are linked in recommender systems and whether disentanglement improves performance. It conducts a reproducibility study across four datasets and five models, quantifying disentanglement with DCI-based metrics and interpreting representations with global LIME/SHAP, then analyzes correlations with effectiveness. The findings show no consistent link between disentanglement and recommendation accuracy, but a robust association between disentanglement and representation interpretability, while reproducibility varies with dataset and ground-truth factor construction. The work highlights the importance of objective disentanglement evaluation, provides public code, and suggests that disentanglement may primarily enhance interpretability rather than performance in RSs.

Abstract

Unsupervised learning of disentangled representations has been closely tied to enhancing the representation intepretability of Recommender Systems (RSs). This has been achieved by making the representation of individual features more distinctly separated, so that it is easier to attribute the contribution of features to the model's predictions. However, such advantages in interpretability and feature attribution have mainly been explored qualitatively. Moreover, the effect of disentanglement on the model's recommendation performance has been largely overlooked. In this work, we reproduce the recommendation performance, representation disentanglement and representation interpretability of five well-known recommendation models on four RS datasets. We quantify disentanglement and investigate the link of disentanglement with recommendation effectiveness and representation interpretability. While several existing work in RSs have proposed disentangled representations as a gateway to improved effectiveness and interpretability, our findings show that disentanglement is not necessarily related to effectiveness but is closely related to representation interpretability. Our code and results are publicly available at https://github.com/edervishaj/disentanglement-interpretability-recsys.

Are Representation Disentanglement and Interpretability Linked in Recommendation Models? A Critical Review and Reproducibility Study

TL;DR

The paper investigates whether representation disentanglement and interpretability are linked in recommender systems and whether disentanglement improves performance. It conducts a reproducibility study across four datasets and five models, quantifying disentanglement with DCI-based metrics and interpreting representations with global LIME/SHAP, then analyzes correlations with effectiveness. The findings show no consistent link between disentanglement and recommendation accuracy, but a robust association between disentanglement and representation interpretability, while reproducibility varies with dataset and ground-truth factor construction. The work highlights the importance of objective disentanglement evaluation, provides public code, and suggests that disentanglement may primarily enhance interpretability rather than performance in RSs.

Abstract

Unsupervised learning of disentangled representations has been closely tied to enhancing the representation intepretability of Recommender Systems (RSs). This has been achieved by making the representation of individual features more distinctly separated, so that it is easier to attribute the contribution of features to the model's predictions. However, such advantages in interpretability and feature attribution have mainly been explored qualitatively. Moreover, the effect of disentanglement on the model's recommendation performance has been largely overlooked. In this work, we reproduce the recommendation performance, representation disentanglement and representation interpretability of five well-known recommendation models on four RS datasets. We quantify disentanglement and investigate the link of disentanglement with recommendation effectiveness and representation interpretability. While several existing work in RSs have proposed disentangled representations as a gateway to improved effectiveness and interpretability, our findings show that disentanglement is not necessarily related to effectiveness but is closely related to representation interpretability. Our code and results are publicly available at https://github.com/edervishaj/disentanglement-interpretability-recsys.

Paper Structure

This paper contains 20 sections, 1 equation, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Disentangled Representations
  4. Disentangled Representations in RSs
  5. Disentanglement, Interpretability of Representations and Effectiveness
  6. Experiments
  7. Measures
  8. Recommendation effectiveness measures
  9. Disentanglement measures
  10. Interpretability measures
  11. Datasets
  12. Sampling and splits
  13. Ground truth factors
  14. Recommendation Models
  15. Results and Discussion
  16. ...and 5 more sections

Figures (2)

  • Figure 1: Repeated measurements correlation of effectiveness, disentanglement and representation interpretability measures for each dataset. (*) denotes statistical significance at $\mathbf{p < 0.05}$.
  • Figure 2: Repeated measurements correlation of effectiveness, disentanglement and representation interpretability measures for each model. (*) denotes statistical significance at $\mathbf{p < 0.05}$.