RecMind: Japanese Movie Recommendation Dialogue with Seeker's Internal State

TL;DR

This work addresses the lack of annotated data for modeling a seeker's internal state in dialogue-based recommendations. It introduces RecMind, a Japanese movie recommendation dialogue dataset with entity-level annotations for subjective ( seeker) and objective (recommender) knowledge and interest across High/Neutral/Low, accompanied by a web-based data collection system and a movie search tool. Through analysis, it shows that entities the seeker lacks knowledge about but is interested in can drive successful recommendations, motivating a Chain-of-Thought prompting framework that explicitly estimates internal state before response generation. GPT-4 based experiments demonstrate that state-aware generation yields more consistent and tailored recommendations than a baseline, with human evaluators noting improvements in consistency, relevance, and success, underscoring RecMind’s value for future internal-state estimation and knowledge-grounded dialogue research.

Abstract

Humans pay careful attention to the interlocutor's internal state in dialogues. For example, in recommendation dialogues, we make recommendations while estimating the seeker's internal state, such as his/her level of knowledge and interest. Since there are no existing annotated resources for the analysis, we constructed RecMind, a Japanese movie recommendation dialogue dataset with annotations of the seeker's internal state at the entity level. Each entity has a subjective label annotated by the seeker and an objective label annotated by the recommender. RecMind also features engaging dialogues with long seeker's utterances, enabling a detailed analysis of the seeker's internal state. Our analysis based on RecMind reveals that entities that the seeker has no knowledge about but has an interest in contribute to recommendation success. We also propose a response generation framework that explicitly considers the seeker's internal state, utilizing the chain-of-thought prompting. The human evaluation results show that our proposed method outperforms the baseline method in both consistency and the success of recommendations.

Figures (4)

• Figure 1: Relationship between recommendation success score and the ratio of each internal state label.
• Figure 2: Overview of our proposed method. The internal state estimation, which is highlighted, is performed only for the proposed method and not for the baseline method.
• Figure 3: Screenshot of the recommender's chatroom
• Figure 4: Screenshot of the seeker's chatroom