Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Chain-of-Conceptual-Thought Elicits Daily Conversation in Large Language Models

Qingqing Gu, Dan Wang, Yue Zhao, Xiaoyu Wang, Zhonglin Jiang, Yong Chen, Hongyan Li, Luo Ji

TL;DR

This work introduces Chain-of-Conceptual Thoughts (CoCT), a prompt-based paradigm that prompts large language models to first output a sequence of concept tags (emotion, strategy, or topic) and then ground the subsequent content in those concepts, enabling inner-utterance concept transitions within a single response. CoCT can operate with pre-defined concept lists or generate concepts on-the-fly, and it supports open-domain daily conversations as well as emotional-support contexts. Across both in-domain and out-of-domain settings, CoCT outperforms strong prompt-based baselines (Direct inference, Self-Refine, ECoT, SoT, Plan-and-Solve, RAG) according to automatic metrics, human judgments, and LLM-based evaluations, while demonstrating robust generalization to new concepts and queries. The findings suggest that inducing a hierarchical, concept-driven thinking process in LLMs enhances response quality in open-domain tasks and offers a scalable prompting strategy that can be integrated with finetuning and external knowledge sources in future work.

Abstract

Chain-of-Thought (CoT) is widely applied to enhance the LLM capability in math, coding and reasoning tasks. However, its performance is limited for open-domain tasks, when there are no clearly defined reasoning steps or logical transitions. To mitigate such challenges, we propose a new prompt-based paradigm called Chain of Conceptual Thoughts (CoCT), which suggests the LLM first to produce the tag of concepts, then complete the detailed content following the concept. To encourage this hierarchical way of thinking, we implement the concepts with emotions, strategies and topics. We experiment with this paradigm in daily and emotional support conversations, covering tasks with both in-domain and out-of-domain concept settings. Automatic, human, and LLM-based evaluations reveal that CoCT surpasses several prompt-based baselines such as self-refine, ECoT, SoT and RAG, suggesting a potential solution of LLM prompting paradigm for a wider scope of tasks.

Chain-of-Conceptual-Thought Elicits Daily Conversation in Large Language Models

TL;DR

This work introduces Chain-of-Conceptual Thoughts (CoCT), a prompt-based paradigm that prompts large language models to first output a sequence of concept tags (emotion, strategy, or topic) and then ground the subsequent content in those concepts, enabling inner-utterance concept transitions within a single response. CoCT can operate with pre-defined concept lists or generate concepts on-the-fly, and it supports open-domain daily conversations as well as emotional-support contexts. Across both in-domain and out-of-domain settings, CoCT outperforms strong prompt-based baselines (Direct inference, Self-Refine, ECoT, SoT, Plan-and-Solve, RAG) according to automatic metrics, human judgments, and LLM-based evaluations, while demonstrating robust generalization to new concepts and queries. The findings suggest that inducing a hierarchical, concept-driven thinking process in LLMs enhances response quality in open-domain tasks and offers a scalable prompting strategy that can be integrated with finetuning and external knowledge sources in future work.

Abstract

Chain-of-Thought (CoT) is widely applied to enhance the LLM capability in math, coding and reasoning tasks. However, its performance is limited for open-domain tasks, when there are no clearly defined reasoning steps or logical transitions. To mitigate such challenges, we propose a new prompt-based paradigm called Chain of Conceptual Thoughts (CoCT), which suggests the LLM first to produce the tag of concepts, then complete the detailed content following the concept. To encourage this hierarchical way of thinking, we implement the concepts with emotions, strategies and topics. We experiment with this paradigm in daily and emotional support conversations, covering tasks with both in-domain and out-of-domain concept settings. Automatic, human, and LLM-based evaluations reveal that CoCT surpasses several prompt-based baselines such as self-refine, ECoT, SoT and RAG, suggesting a potential solution of LLM prompting paradigm for a wider scope of tasks.
Paper Structure (39 sections, 6 equations, 4 figures, 14 tables)

This paper contains 39 sections, 6 equations, 4 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Chain-of-Conceptual-Thought Prompt
  4. Formulation of Concepts
  5. CoCT Elicits the Inner-utterance Transitions
  6. Experiment
  7. Setting
  8. Implementation.
  9. Datasets and tasks.
  10. Metrics.
  11. Baselines.
  12. Results on Daily Conversation
  13. Performance on in-domain concepts.
  14. Generalize with out-of-domain concepts or queries.
  15. Cases on more situations
  16. ...and 24 more sections

Figures (4)

  • Figure 1: Comparison between CoCT and other prompt-based baselines on ESconv.
  • Figure 2: The paradigm of CoCT. The red bold text represents emotions, the blue bold text represents strategies, and the green bold text represents topics of conversation transitions.
  • Figure 3: Comparative visualizations between outer-utterance transitions (Left) and inner-utterance transitions (Right). The dash arrows indicate the conceptual transitions between emotions, strategies or topics.
  • Figure 4: Distribution of conceptual transitions within the utterance. Left: the strategy transition on ESConv. Right: the emotion transition on DailyDialogue.