BMLP: Behavior-aware MLP for Heterogeneous Sequential Recommendation

TL;DR

This work tackles heterogeneous sequential recommendation by modeling multiple user behaviors (e.g., clicks, purchases) with a pure MLP architecture. The proposed BMLP architecture comprises two modules: HIP captures multi-granularity behavior interactions via behavior types and transitions, while PIP adaptively fuses auxiliary behaviors to infer purchase intent; a gating mechanism then merges these signals for final prediction. Empirical results on four public datasets show that BMLP achieves significant improvements over state-of-the-art baselines, with the pure-MLP design offering linear time complexity and practical efficiency. The approach provides a scalable, effective framework for real-world recommender systems that need to leverage diverse user behaviors. Key mathematical relationships include the heterogeneous embedding construction $\mathbf{x}_u^t = \text{concat}(M_u^t, V_{i_u^t})$ with $M_u^t = B_{b_u^t} + \text{trans}(b_u^t, b_u^{t+1})$, and the final prediction combining global and local intents via a gating mechanism, optimized with a cross-entropy-like loss $\mathcal{L} = -\sum_{s}\sum_{j} y_{sj} \log(\hat{r}_{t+1,j}) + (1-y_{sj}) \log(1-\hat{r}_{t+1,j})$, where all mathematical expressions are denoted in $...$.

Abstract

In real recommendation scenarios, users often have different types of behaviors, such as clicking and buying. Existing research methods show that it is possible to capture the heterogeneous interests of users through different types of behaviors. However, most multi-behavior approaches have limitations in learning the relationship between different behaviors. In this paper, we propose a novel multilayer perceptron (MLP)-based heterogeneous sequential recommendation method, namely behavior-aware multilayer perceptron (BMLP). Specifically, it has two main modules, including a heterogeneous interest perception (HIP) module, which models behaviors at multiple granularities through behavior types and transition relationships, and a purchase intent perception (PIP) module, which adaptively fuses subsequences of auxiliary behaviors to capture users' purchase intent. Compared with mainstream sequence models, MLP is competitive in terms of accuracy and has unique advantages in simplicity and efficiency. Extensive experiments show that BMLP achieves significant improvement over state-of-the-art algorithms on four public datasets. In addition, its pure MLP architecture leads to a linear time complexity.

Figures (5)

• Figure 1: The structure of the Behavior-aware MLP (BMLP) is divided into Heterogeneous Interest Perception (HIP), Purchase Intent Perception (PIP), and recommendation module. Details of the behavior replacement block and the Feature Capture Block (FCB) are shown in the right panel. The Sequence Capture Block (SCB) is a special case of FCB.
• Figure 2: The effectiveness of four different approaches of modeling the behavior types in our BMLP on four datasets. Notice that 'S', 'B', and 'T' denote modeling the sequences without the behavior types, with behavior types, and with behavior transition relationships, respectively.
• Figure 3: Recommendation performance in predicting purchased items versus un-purchased items on Rec15, Tmall, and UB. Notice that the behaviors in ML1M are simulated, which are thus not included.
• Figure 4: Recommendation performance of our BMLP with different numbers of heads on four datasets.
• Figure 5: Recommendation performance of our BMLP with different lengths of the most recent auxiliary behavior subsequences, i.e., $L' \in \{3,4,5,6,7\}$, on four datasets.