Have We Really Understood Collaborative Information? An Empirical Investigation

TL;DR

This paper addresses the lack of a quantitative understanding of collaborative information in recommender systems by defining collaborative information through item co-occurrence patterns and a hop-based CR framework. It formulates the sequential recommendation problem, clarifies the key properties of collaborative information (Transitivity, Hierarchy, Redundancy), and introduces a shortest-path-based quantitative definition that distinguishes 0-hop, 1-hop, and higher-order CRs. Through six benchmark datasets, it shows that indirect CRs dominate training-time item-item relations while direct CRs remain rare but highly actionable, and it evaluates a wide range of models (traditional, neural, and LLM-based) to reveal how different CR types influence performance, with neural models and multi-graph approaches like MSGIFSR excelling on complex CRs and LLMs showing stability across CR types. The findings highlight challenges in handling imbalanced and complex CRs and suggest future directions, including integrating collaborative knowledge into LLMs, to develop more effective CR-aware recommender systems.

Abstract

Collaborative information serves as the cornerstone of recommender systems which typically focus on capturing it from user-item interactions to deliver personalized services. However, current understanding of this crucial resource remains limited. Specifically, a quantitative definition of collaborative information is missing, its manifestation within user-item interactions remains unclear, and its impact on recommendation performance is largely unknown. To bridge this gap, this work conducts a systematic investigation of collaborative information. We begin by clarifying collaborative information in terms of item co-occurrence patterns, identifying its main characteristics, and presenting a quantitative definition. We then estimate the distribution of collaborative information from several aspects, shedding light on how collaborative information is structured in practice. Furthermore, we evaluate the impact of collaborative information on the performance of various recommendation algorithms. Finally, we highlight challenges in effectively capturing collaborative information and outlook promising directions for future research. By establishing an empirical framework, we uncover many insightful observations that advance our understanding of collaborative information and offer valuable guidelines for developing more effective recommender systems.

Figures (7)

• Figure 1: A quantitative definition of collaborative information. Diverse collaborative relations are derived based on the complexity of item co-occurrence patterns.
• Figure 2: Distribution of collaborative relations among items.
• Figure 3: Collaborative relation distribution relevant to label items within various benchmarks.
• Figure 4: Item co-occurrence frequency within 0-hop CR on various benchmarks.
• Figure 5: Impact of diverse CRs on RS.