Personalized Multi-task Training for Recommender System

TL;DR

PMTRec tackles personalized multi-task learning in recommender systems by computing per-user gradient contributions from multiple auxiliary tasks and combining them with a gradient-aware approach. It introduces Task Focusing to progressively emphasize the main RecSys objective and Gradient Magnitude Balancing to normalize contributions from tasks with different gradient scales. The method uses a MF-based encoder and auxiliary losses derived from social, co-viewing, or category information to enrich embeddings. Experiments on three real-world datasets show PMTRec outperforms existing multi-task baselines, demonstrating the value of personalized gradient combination in enhancing recommendation accuracy. The work also provides an open-source implementation for further research.

Abstract

In the vast landscape of internet information, recommender systems (RecSys) have become essential for guiding users through a sea of choices aligned with their preferences. These systems have applications in diverse domains, such as news feeds, game suggestions, and shopping recommendations. Personalization is a key technique in RecSys, where modern methods leverage representation learning to encode user/item interactions into embeddings, forming the foundation for personalized recommendations. However, integrating information from multiple sources to enhance recommendation performance remains challenging. This paper introduces a novel approach named PMTRec, the first personalized multi-task learning algorithm to obtain comprehensive user/item embeddings from various information sources. Addressing challenges specific to personalized RecSys, we develop modules to handle personalized task weights, diverse task orientations, and variations in gradient magnitudes across tasks. PMTRec dynamically adjusts task weights based on gradient norms for each user/item, employs a Task Focusing module to align gradient combinations with the main recommendation task, and uses a Gradient Magnitude Balancing module to ensure balanced training across tasks. Through extensive experiments on three real-world datasets with different scales, we demonstrate that PMTRec significantly outperforms existing multi-task learning methods, showcasing its effectiveness in achieving enhanced recommendation accuracy by leveraging multiple tasks simultaneously. Our contributions open new avenues for advancing personalized multi-task training in recommender systems.

Figures (6)

• Figure 1: A toy example of multi-task learning in RecSys.
• Figure 2: Model Framework. Our proposed PMTRec is illustrated in detail with two propagation stages. (1) Forward propagation computes the user/item embedding and the losses for different tasks. (2) Backward propagation pools the gradient obtained from each task with a personalized gradient combination for each user/item. The combined gradient is then used to update the embedding with an optimizer. (Best viewed in color)
• Figure 3: Experiments on different encoders
• Figure 4: Ablation study of PMTRec. Variant 1 omits the task focusing module, Variant 2 excludes the gradient magnitude balancing module, and Variant 3 represents the full PMTRec.
• Figure 5: Sensitivity analysis of PMTRec's hyper-parameters: $\alpha$ is the increment base in the task focusing module, while $\tau$ balances the gradient magnitude in the balancing module.