Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval

Kazuaki Furumai, Roberto Legaspi, Julio Vizcarra, Yudai Yamazaki, Yasutaka Nishimura, Sina J. Semnani, Kazushi Ikeda, Weiyan Shi, Monica S. Lam

TL;DR

The proposed PersuaBot is a zero-shot chatbot based on Large Language Models (LLMs) that is factual and more persuasive by leveraging many more nuanced strategies, while achieving factual accuracy surpassing state-of-the-art knowledge-oriented chatbots.

Abstract

Persuasion plays a pivotal role in a wide range of applications from health intervention to the promotion of social good. Persuasive chatbots employed responsibly for social good can be an enabler of positive individual and social change. Existing methods rely on fine-tuning persuasive chatbots with task-specific training data which is costly, if not infeasible, to collect. Furthermore, they employ only a handful of pre-defined persuasion strategies. We propose PersuaBot, a zero-shot chatbot based on Large Language Models (LLMs) that is factual and more persuasive by leveraging many more nuanced strategies. PersuaBot uses an LLM to first generate natural responses, from which the strategies used are extracted. To combat hallucination of LLMs, Persuabot replace any unsubstantiated claims in the response with retrieved facts supporting the extracted strategies. We applied our chatbot, PersuaBot, to three significantly different domains needing persuasion skills: donation solicitation, recommendations, and health intervention. Our experiments on simulated and human conversations show that our zero-shot approach is more persuasive than prior work, while achieving factual accuracy surpassing state-of-the-art knowledge-oriented chatbots.

Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval

TL;DR

The proposed PersuaBot is a zero-shot chatbot based on Large Language Models (LLMs) that is factual and more persuasive by leveraging many more nuanced strategies, while achieving factual accuracy surpassing state-of-the-art knowledge-oriented chatbots.

Abstract

Persuasion plays a pivotal role in a wide range of applications from health intervention to the promotion of social good. Persuasive chatbots employed responsibly for social good can be an enabler of positive individual and social change. Existing methods rely on fine-tuning persuasive chatbots with task-specific training data which is costly, if not infeasible, to collect. Furthermore, they employ only a handful of pre-defined persuasion strategies. We propose PersuaBot, a zero-shot chatbot based on Large Language Models (LLMs) that is factual and more persuasive by leveraging many more nuanced strategies. PersuaBot uses an LLM to first generate natural responses, from which the strategies used are extracted. To combat hallucination of LLMs, Persuabot replace any unsubstantiated claims in the response with retrieved facts supporting the extracted strategies. We applied our chatbot, PersuaBot, to three significantly different domains needing persuasion skills: donation solicitation, recommendations, and health intervention. Our experiments on simulated and human conversations show that our zero-shot approach is more persuasive than prior work, while achieving factual accuracy surpassing state-of-the-art knowledge-oriented chatbots.
Paper Structure (33 sections, 22 figures, 7 tables)

This paper contains 33 sections, 22 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Our Approach
  4. Contributions
  5. Related Work
  6. Factual Chatbots
  7. Strategizing for persuasive chatbot
  8. Discussion
  9. Method: PersuaBot
  10. Strategy Maintenance Module
  11. Strategy Extraction
  12. Fact-Check & IR
  13. Question Handling Module
  14. Merging Results
  15. Experimental Setup
  16. ...and 18 more sections

Figures (22)

  • Figure 1: An overview of PersuaBot and an example of a conversation to persuade a user to donate to "Save the Children". To generate a factually correct response and maintain its persuasive function, PersuaBot has a Strategy Maintenance Module that (1) generates an LLM response and decomposes it according to its strategy intent, (2) extracts the strategies for each section, (3) fact-checks the sections and uses IR to substantiate the strategies. Additionally, (4) it retrieves relevant information in response to users' requests if any, and (5) merges the results.
  • Figure 2: Part of a conversation between PersuaBot and a tough user in the social good task. The base LLM is GPT-3.5.
  • Figure 3: Part of a conversation between PersuaBot and a tough user in the recommendation task. The base LLM is GPT-3.5.
  • Figure 4: Part of a conversation between PersuaBot and a real user in the social good task. The base LLM is GPT-3.5.
  • Figure 5: A prompt for strategy extraction
  • ...and 17 more figures