Beyond Models! Explainable Data Valuation and Metric Adaption for Recommendation

TL;DR

This work tackles data quality heterogeneity in recommendations by introducing DVR, an interpretable framework that jointly optimizes data valuation and metric adaptation. It introduces a Shapley-value based data valuator, computed efficiently via Harsanyi-interaction networks, to quantify each training sample’s contribution, and a reinforcement-learning driven metric adapter to handle both differentiable and non-differentiable evaluation metrics through bilevel optimization. The approach enables end-to-end data selection and model training that improves accuracy, diversity, and fairness across multiple datasets and backbones, with up to $34.7\%$ NDCG improvements reported. By providing transparent data valuations and versatile metric optimization, DVR enhances reliability and generality for recommender systems in realistic, metric-driven deployment contexts.

Abstract

User behavior records serve as the foundation for recommender systems. While the behavior data exhibits ease of acquisition, it often suffers from varying quality. Current methods employ data valuation to discern high-quality data from low-quality data. However, they tend to employ black-box design, lacking transparency and interpretability. Besides, they are typically tailored to specific evaluation metrics, leading to limited generality across various tasks. To overcome these issues, we propose an explainable and versatile framework DVR which can enhance the efficiency of data utilization tailored to any requirements of the model architectures and evaluation metrics. For explainable data valuation, a data valuator is presented to evaluate the data quality via calculating its Shapley value from the game-theoretic perspective, ensuring robust mathematical properties and reliability. In order to accommodate various evaluation metrics, including differentiable and non-differentiable ones, a metric adapter is devised based on reinforcement learning, where a metric is treated as the reinforcement reward that guides model optimization. Extensive experiments conducted on various benchmarks verify that our framework can improve the performance of current recommendation algorithms on various metrics including ranking accuracy, diversity, and fairness. Specifically, our framework achieves up to 34.7\% improvements over existing methods in terms of representative NDCG metric. The code is available at https://github.com/renqii/DVR.

Figures (2)

• Figure 1: The architecture of the whole DVR framework. The left side of the figure is batch samples from behavior data as the input of the framework. The framework consists of the inner optimization for the recommendation model and the outer optimization of the data valuator. The yellow and blue glowing two-bend arrows mean the backward propagation to optimize the recommendation model and reinforcement learning to optimize the data valuator, respectively.
• Figure 2: The Cosine Similarity between Negative BPR Loss and Shapley Value.