Residual Multi-Task Learner for Applied Ranking

TL;DR

ResFlow is introduced, a lightweight multi-task learning framework that enables efficient cross-task information sharing via residual connections between corresponding layers of task networks and proposes to fuse scores from the multiple tasks additively when ranking items, which outperforms traditional multiplicative fusion.

Abstract

Modern e-commerce platforms rely heavily on modeling diverse user feedback to provide personalized services. Consequently, multi-task learning has become an integral part of their ranking systems. However, existing multi-task learning methods encounter two main challenges: some lack explicit modeling of task relationships, resulting in inferior performance, while others have limited applicability due to being computationally intensive, having scalability issues, or relying on strong assumptions. To address these limitations and better fit our real-world scenario, pre-rank in Shopee Search, we introduce in this paper ResFlow, a lightweight multi-task learning framework that enables efficient cross-task information sharing via residual connections between corresponding layers of task networks. Extensive experiments on datasets from various scenarios and modalities demonstrate its superior performance and adaptability over state-of-the-art methods. The online A/B tests in Shopee Search showcase its practical value in large-scale industrial applications, evidenced by a 1.29% increase in OPU (order-per-user) without additional system latency. ResFlow is now fully deployed in the pre-rank module of Shopee Search. To facilitate efficient online deployment, we propose a novel offline metric Weighted Recall@K, which aligns well with our online metric OPU, addressing the longstanding online-offline metric misalignment issue. Besides, we propose to fuse scores from the multiple tasks additively when ranking items, which outperforms traditional multiplicative fusion. The code is released at https://github.com/BrunoTruthAlliance/ResFlow

Figures (8)

• Figure 1: An illustration of the cascading item filtering process by the multi-stage candidate selection framework in large-scale applied ranking systems and subsequent user feedback. It sketches the quantities of samples at each stage, highlighting the sparsity of conversion feedback.
• Figure 2: The conceptual architectures of ResFlow and its main competitors. Illustrated with three sequentially dependent tasks: CTR, the click-through rate; ATCR, the post-view add-to-cart rate; CTCVR, the click-through & conversion rate. AITM extracts information from the last feature layer of the former task, transferring it to the latter via an attention-based module. ESMM models the (conditional) probability of each step, estimating ATCR and CTCVR based on the products of these probabilities. ResFlow builds residual connections between corresponding layers among tasks, enabling information transfer sufficiently at various levels in an additive manner, which is not only extremely lightweight but also shows distinguished effectiveness.
• Figure 3: Parameter sensitivity experiment results. From left to right: (a) embedding dimension, (b) learning rate, (c) positive sample weight of CTR task, and (d) positive sample weight of CTCVR task. The negative sample weight is fixed as 1.
• Figure 4: Case study of the learned residual. We randomly selected 10 users and sampled 3 items of different types for each user, forming 3 groups: A (click=1, order=1), B (click=1, order=0), and C (click=0, order=0). The learned residual features and logits of the CTCVR tower, and other related, are shown. Trained on the AliCCP dataset. ResFlow tends to learn less extreme features than NSE. The learned residual logits are all negative, and the values in group B are generally smaller than those in group A.
• Figure 5: Visualization of different task topologies of KuaiRand-Pure-S1 Multi-Task.