Generative Data Transformation: From Mixed to Unified Data

TL;DR

Experiments show \textsc{Taesar} outperforms model-centric solutions and generalizes to various sequential models, effectively combines the strengths of data- and model-centric paradigms.

Abstract

Recommendation model performance is intrinsically tied to the quality, volume, and relevance of their training data. To address common challenges like data sparsity and cold start, recent researchs have leveraged data from multiple auxiliary domains to enrich information within the target domain. However, inherent domain gaps can degrade the quality of mixed-domain data, leading to negative transfer and diminished model performance. Existing prevailing \emph{model-centric} paradigm -- which relies on complex, customized architectures -- struggles to capture the subtle, non-structural sequence dependencies across domains, leading to poor generalization and high demands on computational resources. To address these shortcomings, we propose \textsc{Taesar}, a \emph{data-centric} framework for \textbf{t}arget-\textbf{a}lign\textbf{e}d \textbf{s}equenti\textbf{a}l \textbf{r}egeneration, which employs a contrastive decoding mechanism to adaptively encode cross-domain context into target-domain sequences. It employs contrastive decoding to encode cross-domain context into target sequences, enabling standard models to learn intricate dependencies without complex fusion architectures. Experiments show \textsc{Taesar} outperforms model-centric solutions and generalizes to various sequential models. By generating enriched datasets, \textsc{Taesar} effectively combines the strengths of data- and model-centric paradigms. The code accompanying this paper is available at~ \textcolor{blue}{https://github.com/USTC-StarTeam/Taesar}.

Figures (5)

• Figure 1: Cosine similarity heatmap of gradient directions across four domains in multi-domain sequential recommendation. Naïvely combining data from multiple domains for training induces gradient conflicts and inconsistencies.
• Figure 2: Motivation of $\textsc{Taesar}$. We aim to eliminate the domain semantic gap at the source level, prior to model training, thereby transforming cross-domain mixed data into unified target-domain data. Left: Source items with high transferability are mapped to semantically closest target items, while others are discarded. Right: Regenerated data aligns cross-domain behavioral patterns with target-domain relevance, improving model gradients and target-domain performance.
• Figure 3: Overview of $\textsc{Taesar}$. (1) In the first stage, Tri-model Pretraining, we construct three views of decoder models: a base model ($\theta_{\mathcal{M}}$), distinct source domain experts ($\theta_{\mathcal{S}}$), and a target domain expert ($\theta_{\mathcal{T}}$). (2) We select the base model, the target domain expert, and the source domain expert corresponding to the item to be transformed to regenerate mixed cross-domain sequences using global and local contrastive decoding among the three models. (3) Based on the two types of contrastive scores, we decide whether to replace a source-domain item with a target-domain item or to discard the source-domain item entirely.
• Figure 4: Schematic illustration of the adaptive contrastive decoding phase: (a) selection of transferable items from source domain $\mathcal{S}$ (Electronics), and (b) distribution-level mapping to target domain $\mathcal{T}$ (Books).
• Figure 5: Comparison of data statistics between the original and regenerated Books domain dataset: (a) item frequency, (b) long-tail coverage, and (c) sequence length distribution.