A Preference-oriented Diversity Model Based on Mutual-information in Re-ranking for E-commerce Search
Huimu Wang, Mingming Li, Dadong Miao, Songlin Wang, Guoyu Tang, Lin Liu, Sulong Xu, Jinghe Hu
TL;DR
The paper addresses the challenge of balancing accuracy and diversity in e-commerce re-ranking. It introduces PODM-MI, a dual-component framework consisting of a preference-oriented network (PON) that models user diversity and item diversity via multi-dimensional Gaussian distributions, and a self-adaptive model (SAM) that maximizes the mutual information between user preferences and item distributions using a variational lower bound. A learnable utility matrix then fuses this alignment into adaptive ranking, enabling joint optimization of relevance and novelty. Experimental results on a large JD.com dataset show consistent offline improvements in accuracy and diversity, with significant online gains in user conversions and engagement, culminating in successful deployment and clear business impact.
Abstract
Re-ranking is a process of rearranging ranking list to more effectively meet user demands by accounting for the interrelationships between items. Existing methods predominantly enhance the precision of search results, often at the expense of diversity, leading to outcomes that may not fulfill the varied needs of users. Conversely, methods designed to promote diversity might compromise the precision of the results, failing to satisfy the users' requirements for accuracy. To alleviate the above problems, this paper proposes a Preference-oriented Diversity Model Based on Mutual-information (PODM-MI), which consider both accuracy and diversity in the re-ranking process. Specifically, PODM-MI adopts Multidimensional Gaussian distributions based on variational inference to capture users' diversity preferences with uncertainty. Then we maximize the mutual information between the diversity preferences of the users and the candidate items using the maximum variational inference lower bound to enhance their correlations. Subsequently, we derive a utility matrix based on the correlations, enabling the adaptive ranking of items in line with user preferences and establishing a balance between the aforementioned objectives. Experimental results on real-world online e-commerce systems demonstrate the significant improvements of PODM-MI, and we have successfully deployed PODM-MI on an e-commerce search platform.