Personalization of Dataset Retrieval Results using a Metadata-based Data Valuation Method

TL;DR

The paper tackles the problem of improving dataset retrieval by introducing a metadata-based data valuation method that personalizes rankings with user-provided weights over four metadata dimensions. It formalizes dataset value as $V(d_i)=w_U\cdot U_i + w_S\cdot S_i + w_C\cdot C_i + w_O\cdot O_i$, normalizes inputs, and models data currency for creation date with $Q_{Curr.}(\omega,A)=\exp(-0.2\cdot age(\omega,A))$, enabling personalized, interactive ranking in a national-mapping-agency use case. Experimental results evaluated with $NDCG$ and $NDCG@5$ show promising possibilities (e.g., $NDCG@5$ up to 0.8207 for SH2 with simple averaging) but also indicate variability across stakeholders and methods, highlighting the need for further refinement and broader validation. The work demonstrates the potential of data valuation to tailor dataset retrieval to user preferences, enabling more effective data discovery in large catalogs and data-rich environments.

Abstract

In this paper, we propose a novel data valuation method for a Dataset Retrieval (DR) use case in Ireland's National mapping agency. To the best of our knowledge, data valuation has not yet been applied to Dataset Retrieval. By leveraging metadata and a user's preferences, we estimate the personal value of each dataset to facilitate dataset retrieval and filtering. We then validated the data value-based ranking against the stakeholders' ranking of the datasets. The proposed data valuation method and use case demonstrated that data valuation is promising for dataset retrieval. For instance, the outperforming dataset retrieval based on our approach obtained 0.8207 in terms of NDCG@5 (the truncated Normalized Discounted Cumulative Gain at 5). This study is unique in its exploration of a data valuation-based approach to dataset retrieval and stands out because, unlike most existing methods, our approach is validated using the stakeholders ranking of the datasets.

Figures (1)

• Figure 1: Experimental design for personalized metadata-based data valuation.