Hesitation and Tolerance in Recommender Systems

TL;DR

This work identifies hesitation as an intermediate cognitive state between interest and disinterest in recommender-system interactions and shows how it can evolve into tolerance, a negative experience that reduces user activity and loyalty. Through large-scale surveys (over 6k and 3k responses), offline analyses on Taobao and Kuaishou, and online A/B tests on a major short-video platform, the study demonstrates that tolerance signals degrade engagement but can be harnessed to improve retention when incorporated into training objectives with appropriate labeling. The authors propose a formal framework to classify signals as positive, tolerance, or negative, and validate this approach with two rounds of online experiments showing measurable Day-2 retention gains with minimal additional computation. Overall, the paper provides practical, scalable methods to recognize and leverage nuanced user signals to enhance long-term satisfaction and platform engagement in recommender systems.

Abstract

User interactions in recommender systems are inherently complex, often involving behaviors that go beyond simple acceptance or rejection. One particularly common behavior is hesitation, where users deliberate over recommended items, signaling uncertainty. Our large-scale surveys, with 6,644 and 3,864 responses respectively, confirm that hesitation is not only widespread but also has a profound impact on user experiences. When users spend additional time engaging with content they are ultimately uninterested in, this can lead to negative emotions, a phenomenon we term as tolerance. The surveys reveal that such tolerance behaviors often arise after hesitation and can erode trust, satisfaction, and long-term loyalty to the platform. For instance, a click might reflect a need for more information rather than genuine interest, and prolonged exposure to unsuitable content amplifies frustration. This misalignment between user intent and system interpretation introduces noise into recommendation training, resulting in suggestions that increase uncertainty and disengagement. To address these issues, we identified signals indicative of tolerance behavior and analyzed datasets from both e-commerce and short-video platforms. The analysis shows a strong correlation between increased tolerance behavior and decreased user activity. We integrated these insights into the training process of a recommender system for a major short-video platform. Results from four independent online A/B experiments demonstrated significant improvements in user retention, achieved with minimal additional computational costs. These findings underscore the importance of recognizing hesitation as a ubiquitous user behavior and addressing tolerance to enhance satisfaction, build trust, and sustain long-term engagement in recommender systems.

Figures (6)

• Figure 1: The potential relationship between users' trust in the platform and their interest in its recommended items.
• Figure 2: Demographics of respondents from both surveys include age, highest level of education (completed or in progress), and occupation. User profiles remain largely consistent across both surveys.
• Figure 3: The relationship between users' trust in the platform and their interest in recommended items.
• Figure 4: The process of hesitation (and tolerance) on the recommended item/content.
• Figure 5: Tolerance behaviors in reference week vs proportion of users who show a decrease of engagement in investigation week.