Flow Matching based Sequential Recommender Model

TL;DR

Diffusion-based sequential recommender models suffer from noise perturbations in both forward and reverse steps, which can distort user preferences and produce suboptimal next-item suggestions. FMRec addresses these issues by adopting Flow Matching with a straight trajectory, a modified training target that predicts $f_\Theta(z_t,t)$ toward the true embedding $x_c$, a decoder-based reconstruction loss, and a deterministic ODE-based reverse sampler to eliminate randomness during generation. The approach integrates a robust transformer decoder, regularized losses $\mathcal L_{CE}$ and $\mathcal L_{MSE}$, and mode-sampled timesteps to stabilize learning, achieving improved robustness to noise. Empirical results on four benchmark datasets show FMRec attains an average improvement of 6.53% over state-of-the-art methods, with notable gains over both traditional and diffusion-based baselines, underscoring the practical value of straight-flow diffusion in sequential recommendation.

Abstract

Generative models, particularly diffusion model, have emerged as powerful tools for sequential recommendation. However, accurately modeling user preferences remains challenging due to the noise perturbations inherent in the forward and reverse processes of diffusion-based methods. Towards this end, this study introduces FMRec, a Flow Matching based model that employs a straight flow trajectory and a modified loss tailored for the recommendation task. Additionally, from the diffusion-model perspective, we integrate a reconstruction loss to improve robustness against noise perturbations, thereby retaining user preferences during the forward process. In the reverse process, we employ a deterministic reverse sampler, specifically an ODE-based updating function, to eliminate unnecessary randomness, thereby ensuring that the generated recommendations closely align with user needs. Extensive evaluations on four benchmark datasets reveal that FMRec achieves an average improvement of 6.53% over state-of-the-art methods. The replication code is available at https://github.com/FengLiu-1/FMRec.

Figures (8)

• Figure 1: An illustration that highlights the differences between Diffusion-based (a) and Flow Matching based (b) sequential recommender models in both the forward and reverse processes.
• Figure 2: The framework of the FMRec. In the training process, our design incorporates the development of straight flow trajectories, modifications to the learning target $\mathcal{L}_{FM}$, the design of a decoder-based model, and the implementation of regularized loss functions $\mathcal{L}_{CE}$ and $\mathcal{L}_{MSE}$. In the inference process, we present a deterministic reverse sampler that generates recommendations.
• Figure 3: Performance comparison based on different flow trajectories, measured as percentages(%): "Cosine" represents the results obtained using the Cosine trajectory, while "FMRec" denotes the use of the straight trajectory.
• Figure 4: Comparison of model performance across various $\alpha$ and $\beta$ on the Beauty dataset, measured as percentages(%). These parameters affect the importance of the cross-entropy loss and the reconstruction loss.
• Figure 5: Performance comparison across different values of $\delta$, measured as percentages(%), where $\delta$ controls the weight of noise perturbation in the user's sequential interactions fed into the model.