Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Logic-Scaffolding: Personalized Aspect-Instructed Recommendation Explanation Generation using LLMs

Behnam Rahdari, Hao Ding, Ziwei Fan, Yifei Ma, Zhuotong Chen, Anoop Deoras, Branislav Kveton

TL;DR

The paper tackles the challenge of producing personalized, reliable explanations for recommendations with LLMs. It introduces Logic-Scaffolding, which coordinates relevant-item selection, aspect extraction via few-shot prompts, and chain-of-thought reasoning to generate explanations with intermediate reasoning steps. Empirical evaluation on MovieLens 1M with Falcon-40b demonstrates superior ratings on relevance, readability, factuality, and proper utterance compared to a zero-shot baseline, with statistically significant improvements and large effect sizes. This work advances practical, transparent personalization in explainable recommender systems by enabling domain-aligned aspects and structured reasoning within LLM-based explanations.

Abstract

The unique capabilities of Large Language Models (LLMs), such as the natural language text generation ability, position them as strong candidates for providing explanation for recommendations. However, despite the size of the LLM, most existing models struggle to produce zero-shot explanations reliably. To address this issue, we propose a framework called Logic-Scaffolding, that combines the ideas of aspect-based explanation and chain-of-thought prompting to generate explanations through intermediate reasoning steps. In this paper, we share our experience in building the framework and present an interactive demonstration for exploring our results.

Logic-Scaffolding: Personalized Aspect-Instructed Recommendation Explanation Generation using LLMs

TL;DR

The paper tackles the challenge of producing personalized, reliable explanations for recommendations with LLMs. It introduces Logic-Scaffolding, which coordinates relevant-item selection, aspect extraction via few-shot prompts, and chain-of-thought reasoning to generate explanations with intermediate reasoning steps. Empirical evaluation on MovieLens 1M with Falcon-40b demonstrates superior ratings on relevance, readability, factuality, and proper utterance compared to a zero-shot baseline, with statistically significant improvements and large effect sizes. This work advances practical, transparent personalization in explainable recommender systems by enabling domain-aligned aspects and structured reasoning within LLM-based explanations.

Abstract

The unique capabilities of Large Language Models (LLMs), such as the natural language text generation ability, position them as strong candidates for providing explanation for recommendations. However, despite the size of the LLM, most existing models struggle to produce zero-shot explanations reliably. To address this issue, we propose a framework called Logic-Scaffolding, that combines the ideas of aspect-based explanation and chain-of-thought prompting to generate explanations through intermediate reasoning steps. In this paper, we share our experience in building the framework and present an interactive demonstration for exploring our results.
Paper Structure (9 sections, 7 figures, 1 table)

This paper contains 9 sections, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Characteristics of a Good Explanation
  3. Aspect-Instructed Recommendation Evidence Generation
  4. Relevant Item Selection
  5. Aspect Extraction
  6. Chain-of-Thought Reasoning
  7. Demonstration of Results
  8. Generating the Explanation
  9. Human Evaluation

Figures (7)

  • Figure 1: Template used for generating zero-shot explanations. We use cosine distance between movie text embeddings to find the top five relevant recommendations based on user history.
  • Figure 2: The prompt template employed for aspect extraction.
  • Figure 3: Overview of the Logic-Scaffolding framework.
  • Figure 4: The prompt template used for Chain-of-Thought reasoning.
  • Figure 5: The user interface used in our demonstration enables users to explore the effect of our proposed framework on the explanations quality.
  • ...and 2 more figures