A Best-of-Both Approach to Improve Match Predictions and Reciprocal Recommendations for Job Search

TL;DR

This work addresses reciprocal job matching under extreme label sparsity by proposing a best-of-both (BoB) framework that creates dense pseudo-match scores by blending true match labels with dense match predictions. A meta-model is trained to predict final match likelihoods from these pseudo-scores, with the option to personalize the blend weight $\alpha$ at the user or segment level. Offline production experiments show BoB outperforms standard direct prediction and predict-then-aggregate baselines, especially when using personalized $\alpha$ values (e.g., High, Middle, Low activity segments with $\alpha$ values of 0.0, 0.75, 0.75). The approach demonstrates practical improvements in match discovery and offers a pathway to applying personalized pseudo-labeling in other reciprocal domains such as online dating or mentor matching.

Abstract

Matching users with mutual preferences is a critical aspect of services driven by reciprocal recommendations, such as job search. To produce recommendations in such scenarios, one can predict match probabilities and construct rankings based on these predictions. However, this direct match prediction approach often underperforms due to the extreme sparsity of match labels. Therefore, most existing methods predict preferences separately for each direction (e.g., job seeker to employer and employer to job seeker) and then aggregate the predictions to generate overall matching scores and produce recommendations. However, this typical approach often leads to practical issues, such as biased error propagation between the two models. This paper introduces and demonstrates a novel and practical solution to improve reciprocal recommendations in production by leveraging pseudo-match scores. Specifically, our approach generates dense and more directly relevant pseudo-match scores by combining the true match labels, which are accurate but sparse, with relatively inaccurate but dense match predictions. We then train a meta-model to output the final match predictions by minimizing the prediction loss against the pseudo-match scores. Our method can be seen as a best-of-both (BoB) approach, as it combines the high-level ideas of both direct match prediction and the two separate models approach. It also allows for user-specific weights to construct personalized pseudo-match scores, achieving even better matching performance through appropriate tuning of the weights. Offline experiments on real-world job search data demonstrate the superior performance of our BoB method, particularly with personalized pseudo-match scores, compared to existing approaches in terms of finding potential matches.

Figures (2)

• Figure 1: The figure illustrates our problem of reciprocal recommendation. The platform generates a ranking of job seekers for companies, and then companies decide whether to send a scout to the job seekers in the ranking. Each job seeker who receives a scout then decides whether to respond. A successful match occurs only when a response from the job seeker is observed.
• Figure 2: Performance comparison of the BoB method versus the best baseline (Harmonic Mean) for varying $\alpha$ values for each segment. The graphs illustrate the relative improvement in NDCG@10 for the High, Middle, and Low activity segments. The x-axis represents $\alpha$ values ranging from 0 to 1, while the y-axis shows the relative performance compared to the best baseline (represented by the horizontal dashed line at 1.0).