Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Time-aware Continual User Representation Learning

Seungyoon Choi, Sein Kim, Hongseok Kang, Wonjoong Kim, Chanyoung Park

TL;DR

The paper tackles the mismatch between traditional universal user representation learning and real-world dynamics where item distributions evolve over time, causing catastrophic forgetting. It introduces DITTO, a time-aware continual learner that performs distribution-aware forward and backward knowledge transfer to both preserve past knowledge and adapt it to current shifts, under a practical evaluation scenario that reflects time progression. Empirical results across Tmall, MovieLens, and Taobao show that DITTO outperforms state-of-the-art baselines, with robust performance under varying task orders and noise, and it is backbone-agnostic. The work provides a practical framework for deploying universal user representations in dynamic environments and offers insights into bidirectional transfer mechanisms for continual learning in time-evolving settings.

Abstract

Traditional user modeling (UM) approaches have primarily focused on designing models for a single specific task, but they face limitations in generalization and adaptability across various tasks. Recognizing these challenges, recent studies have shifted towards continual learning (CL)-based universal user representation learning aiming to develop a single model capable of handling multiple tasks. Despite advancements, existing methods are in fact evaluated under an unrealistic scenario that does not consider the passage of time as tasks progress, which overlooks newly emerged items that may change the item distribution of previous tasks. In this paper, we introduce a practical evaluation scenario on which CL-based universal user representation learning approaches should be evaluated, which takes into account the passage of time as tasks progress. Then, we propose a novel framework Dynamic Time-aware continual user representation learner, named DITTO, designed to alleviate catastrophic forgetting despite continuous shifts in item distribution, while also allowing the knowledge acquired from previous tasks to adapt to the current shifted item distribution. Through our extensive experiments, we demonstrate the superiority of DITTO over state-of-the-art methods under a practical evaluation scenario. Our source code is available at https://github.com/seungyoon-Choi/DITTO_official.

Dynamic Time-aware Continual User Representation Learning

TL;DR

The paper tackles the mismatch between traditional universal user representation learning and real-world dynamics where item distributions evolve over time, causing catastrophic forgetting. It introduces DITTO, a time-aware continual learner that performs distribution-aware forward and backward knowledge transfer to both preserve past knowledge and adapt it to current shifts, under a practical evaluation scenario that reflects time progression. Empirical results across Tmall, MovieLens, and Taobao show that DITTO outperforms state-of-the-art baselines, with robust performance under varying task orders and noise, and it is backbone-agnostic. The work provides a practical framework for deploying universal user representations in dynamic environments and offers insights into bidirectional transfer mechanisms for continual learning in time-evolving settings.

Abstract

Traditional user modeling (UM) approaches have primarily focused on designing models for a single specific task, but they face limitations in generalization and adaptability across various tasks. Recognizing these challenges, recent studies have shifted towards continual learning (CL)-based universal user representation learning aiming to develop a single model capable of handling multiple tasks. Despite advancements, existing methods are in fact evaluated under an unrealistic scenario that does not consider the passage of time as tasks progress, which overlooks newly emerged items that may change the item distribution of previous tasks. In this paper, we introduce a practical evaluation scenario on which CL-based universal user representation learning approaches should be evaluated, which takes into account the passage of time as tasks progress. Then, we propose a novel framework Dynamic Time-aware continual user representation learner, named DITTO, designed to alleviate catastrophic forgetting despite continuous shifts in item distribution, while also allowing the knowledge acquired from previous tasks to adapt to the current shifted item distribution. Through our extensive experiments, we demonstrate the superiority of DITTO over state-of-the-art methods under a practical evaluation scenario. Our source code is available at https://github.com/seungyoon-Choi/DITTO_official.

Paper Structure

This paper contains 34 sections, 21 equations, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Universal User Representation
  4. Continual Learning
  5. Preliminaries
  6. Problem Formulation
  7. Notations
  8. Task Description
  9. Backbone Structure
  10. Backbone Network $\mathcal{M}$
  11. Task-specific Mask $\mathbf{m}$
  12. A Practical Evaluation Scenario
  13. Detailed Explanation for Experimental Scenario
  14. Proposed Method: DITTO
  15. Forward Knowledge Transfer: Alleviating Catastrophic Forgetting
  16. ...and 19 more sections

Figures (3)

  • Figure 1: Performance of TERACON in scenarios where new items do not emerge (blue) and new items emerge (red) as the task progresses. The next item prediction tasks (Task1-4) are evaluated using MRR@5, while user profile prediction tasks (Task5) is evaluated using Accuracy.
  • Figure 2: Train scenario of Existing Works and DITTO.
  • Figure 3: Overall model framework. (a) User sampling strategy for FKT, BKT$_1$, and BKT$_2$ modules in task $T_i^{t_j}$. (b) FKT from a previous task $T_{k}$. (c) BKT$_1$ to a previous task $T_1$. (d) BKT$_2$ to a previous task $T_{k}$.