Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HiNet: Novel Multi-Scenario & Multi-Task Learning with Hierarchical Information Extraction

Jie Zhou, Xianshuai Cao, Wenhao Li, Lin Bo, Kun Zhang, Chuan Luo, Qian Yu

TL;DR

A Hierarchical information extraction Network (HiNet) for multi-scenario and multi-task recommendation, which achieves hierarchical extraction based on coarse-to-fine knowledge transfer scheme and a novel scenario-aware attentive network module is proposed to model correlations between scenarios explicitly.

Abstract

Multi-scenario & multi-task learning has been widely applied to many recommendation systems in industrial applications, wherein an effective and practical approach is to carry out multi-scenario transfer learning on the basis of the Mixture-of-Expert (MoE) architecture. However, the MoE-based method, which aims to project all information in the same feature space, cannot effectively deal with the complex relationships inherent among various scenarios and tasks, resulting in unsatisfactory performance. To tackle the problem, we propose a Hierarchical information extraction Network (HiNet) for multi-scenario and multi-task recommendation, which achieves hierarchical extraction based on coarse-to-fine knowledge transfer scheme. The multiple extraction layers of the hierarchical network enable the model to enhance the capability of transferring valuable information across scenarios while preserving specific features of scenarios and tasks. Furthermore, a novel scenario-aware attentive network module is proposed to model correlations between scenarios explicitly. Comprehensive experiments conducted on real-world industrial datasets from Meituan Meishi platform demonstrate that HiNet achieves a new state-of-the-art performance and significantly outperforms existing solutions. HiNet is currently fully deployed in two scenarios and has achieved 2.87% and 1.75% order quantity gain respectively.

HiNet: Novel Multi-Scenario & Multi-Task Learning with Hierarchical Information Extraction

TL;DR

A Hierarchical information extraction Network (HiNet) for multi-scenario and multi-task recommendation, which achieves hierarchical extraction based on coarse-to-fine knowledge transfer scheme and a novel scenario-aware attentive network module is proposed to model correlations between scenarios explicitly.

Abstract

Multi-scenario & multi-task learning has been widely applied to many recommendation systems in industrial applications, wherein an effective and practical approach is to carry out multi-scenario transfer learning on the basis of the Mixture-of-Expert (MoE) architecture. However, the MoE-based method, which aims to project all information in the same feature space, cannot effectively deal with the complex relationships inherent among various scenarios and tasks, resulting in unsatisfactory performance. To tackle the problem, we propose a Hierarchical information extraction Network (HiNet) for multi-scenario and multi-task recommendation, which achieves hierarchical extraction based on coarse-to-fine knowledge transfer scheme. The multiple extraction layers of the hierarchical network enable the model to enhance the capability of transferring valuable information across scenarios while preserving specific features of scenarios and tasks. Furthermore, a novel scenario-aware attentive network module is proposed to model correlations between scenarios explicitly. Comprehensive experiments conducted on real-world industrial datasets from Meituan Meishi platform demonstrate that HiNet achieves a new state-of-the-art performance and significantly outperforms existing solutions. HiNet is currently fully deployed in two scenarios and has achieved 2.87% and 1.75% order quantity gain respectively.
Paper Structure (26 sections, 9 equations, 3 figures, 5 tables)

This paper contains 26 sections, 9 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-Scenario Learning
  4. Multi-task Learning
  5. The HiNet Approach
  6. Problem Formulation
  7. Hierarchical Information Extraction Network
  8. Scenario Extraction Layer
  9. Scenario-shared/specific Expert Network
  10. Scenario-aware Attentive Network
  11. Task Extraction Layer
  12. Customized Gate Control
  13. Training Objective
  14. Experiment
  15. Experiment Setup
  16. ...and 11 more sections

Figures (3)

  • Figure 1: Illustration of user interaction flow inside the Meituan Meishi application. Users may access multiple scenarios listed in the figure. As shown at the bottom, users enter the item details page via the click behavior and then complete the item purchase through the order submission behavior, corresponding to the CTR and CTCVR task in the recommendation system. In addition, the red rectangles represent that the same item may be presented to the user in multiple scenarios.
  • Figure 2: (A) The architecture of the proposed HiNet, which utilizes Scenario Extraction Layer and Task Extraction Layer to obtain scenario and task representations, respectively. (B) At the scenario extraction layer, the Scenario-aware Attentive Network (SAN) module is designed to enhance the representation learning process of current scenario in addition to the scenario sharing/specific experts. The CGC module is adopted at the task extraction layer to further extract task information. (C) The SEI (Sub-Expert Integration) module is used in the scenario extraction layer.
  • Figure 3: Visualization of correlation weights among different scenarios.