HMAR: Hierarchical Masked Attention for Multi-Behaviour Recommendation

TL;DR

HMAR addresses multi‑behavior recommendation by preserving sequential dynamics while modeling multiple user behaviors. It introduces a two‑stage hierarchical masked attention mechanism: a Behavior Encoder that processes items sharing the same behavior with masks, followed by a Sequence Encoder that attends across all behaviors, complemented by Historical Behavior Indicators to encode behavior frequency. The model is trained with multi‑task learning, optimizing a weighted ranking loss $\mathcal{L}_{rank}$ and an auxiliary behavior type loss $\mathcal{L}_{class}$ in the combined objective $\mathcal{L}_{model}=\mathcal{L}_{rank}+\theta\mathcal{L}_{class}$, using autoregressive training. Experiments on four real‑world datasets show HMAR achieves state‑of‑the‑art performance, with ablations confirming the importance of auxiliary behaviors, HBI, and the two‑stage attention. The work advances practical multi‑behavior recommendations and provides code and data for reproducibility in real systems.

Abstract

In the context of recommendation systems, addressing multi-behavioral user interactions has become vital for understanding the evolving user behavior. Recent models utilize techniques like graph neural networks and attention mechanisms for modeling diverse behaviors, but capturing sequential patterns in historical interactions remains challenging. To tackle this, we introduce Hierarchical Masked Attention for multi-behavior recommendation (HMAR). Specifically, our approach applies masked self-attention to items of the same behavior, followed by self-attention across all behaviors. Additionally, we propose historical behavior indicators to encode the historical frequency of each items behavior in the input sequence. Furthermore, the HMAR model operates in a multi-task setting, allowing it to learn item behaviors and their associated ranking scores concurrently. Extensive experimental results on four real-world datasets demonstrate that our proposed model outperforms state-of-the-art methods. Our code and datasets are available here (https://github.com/Shereen-Elsayed/HMAR).

Figures (3)

• Figure 1: Historical behavior indicator example
• Figure 2: Hierarchical Masked Attention Model Architecture
• Figure 3: Behavior Encoder Architecture