ALPBench: A Benchmark for Attribution-level Long-term Personal Behavior Understanding

TL;DR

ALPBench redefines recommendation evaluation by focusing on attribution-level long-term personal behavior understanding, using long-horizon, language-grounded histories to predict coherent attribute combinations rather than single items. The benchmark is built from real-world e-commerce logs across eight categories and three historical horizons, with a formalized task that requires joint reasoning over multiple attributes $\mathcal{A}_C$ and their candidate values $\mathcal{V}_j$ to produce a profile $\hat{\mathbf{y}}$. Extensive zero-shot evaluations across diverse LLM families reveal that while model size improves performance, no model consistently dominates, and profile-level reasoning remains notably difficult, especially as context grows. Limitations include a text-only setup and domain-specific data, pointing to future work on multimodal integration, cross-domain generalization, and practical deployment in recommender systems.

Abstract

Recent advances in large language models have highlighted their potential for personalized recommendation, where accurately capturing user preferences remains a key challenge. Leveraging their strong reasoning and generalization capabilities, LLMs offer new opportunities for modeling long-term user behavior. To systematically evaluate this, we introduce ALPBench, a Benchmark for Attribution-level Long-term Personal Behavior Understanding. Unlike item-focused benchmarks, ALPBench predicts user-interested attribute combinations, enabling ground-truth evaluation even for newly introduced items. It models preferences from long-term historical behaviors rather than users' explicitly expressed requests, better reflecting enduring interests. User histories are represented as natural language sequences, allowing interpretable, reasoning-based personalization. ALPBench enables fine-grained evaluation of personalization by focusing on the prediction of attribute combinations task that remains highly challenging for current LLMs due to the need to capture complex interactions among multiple attributes and reason over long-term user behavior sequences.

Figures (5)

• Figure 1: The complete data collection process of ALPBench, including four stages: data collection, data filtering, data cleaning, and data review.
• Figure 2: Performance Comparison of Different LLMs Across Varying Token Lengths.
• Figure 3: Evaluation prompt template.
• Figure 4: Example of output.
• Figure 5: Attribute Tag Classification Example with Gemini.