Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs

Yu Xia, Rui Wang, Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian McAuley, Shuai Li

TL;DR

Chain-of-X expands Chain-of-Thought by introducing node-level components (intermediates, augmentation, feedback, and models) to tackle diverse tasks with LLMs. The paper defines CoX, surveys representative methods organized by node types and application tasks, and discusses implications and future research directions. It highlights how CoX supports multi-modal reasoning, factuality and safety, and agent-based workflows, while addressing challenges such as inference cost and end-to-end training. The work provides a taxonomy and a comprehensive resource to help researchers apply CoX in broader domains and spur further innovation.

Abstract

Chain-of-Thought (CoT) has been a widely adopted prompting method, eliciting impressive reasoning abilities of Large Language Models (LLMs). Inspired by the sequential thought structure of CoT, a number of Chain-of-X (CoX) methods have been developed to address various challenges across diverse domains and tasks involving LLMs. In this paper, we provide a comprehensive survey of Chain-of-X methods for LLMs in different contexts. Specifically, we categorize them by taxonomies of nodes, i.e., the X in CoX, and application tasks. We also discuss the findings and implications of existing CoX methods, as well as potential future directions. Our survey aims to serve as a detailed and up-to-date resource for researchers seeking to apply the idea of CoT to broader scenarios.

Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs

TL;DR

Chain-of-X expands Chain-of-Thought by introducing node-level components (intermediates, augmentation, feedback, and models) to tackle diverse tasks with LLMs. The paper defines CoX, surveys representative methods organized by node types and application tasks, and discusses implications and future research directions. It highlights how CoX supports multi-modal reasoning, factuality and safety, and agent-based workflows, while addressing challenges such as inference cost and end-to-end training. The work provides a taxonomy and a comprehensive resource to help researchers apply CoX in broader domains and spur further innovation.

Abstract

Chain-of-Thought (CoT) has been a widely adopted prompting method, eliciting impressive reasoning abilities of Large Language Models (LLMs). Inspired by the sequential thought structure of CoT, a number of Chain-of-X (CoX) methods have been developed to address various challenges across diverse domains and tasks involving LLMs. In this paper, we provide a comprehensive survey of Chain-of-X methods for LLMs in different contexts. Specifically, we categorize them by taxonomies of nodes, i.e., the X in CoX, and application tasks. We also discuss the findings and implications of existing CoX methods, as well as potential future directions. Our survey aims to serve as a detailed and up-to-date resource for researchers seeking to apply the idea of CoT to broader scenarios.
Paper Structure (41 sections, 4 figures, 1 table)

This paper contains 41 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Motivation
  3. Distinguishing Focus
  4. Overview of the Survey
  5. What is Chain-of-X?
  6. Chain-of-Thought
  7. Chain-of-X
  8. Chain-of-X Nodes
  9. Chain-of-Intermediates
  10. Problem Decomposition
  11. Knowledge Composition
  12. Chain-of-Augmentation
  13. Instructions
  14. Histories
  15. Retrievals
  16. ...and 26 more sections

Figures (4)

  • Figure 1: Illustrations of Chain-of-X paradigms with four types of nodes: (a) Intermediates, e.g., Thought (§ \ref{['sec:inter']}), (b) Augmentation (§ \ref{['sec:augmt']}), (c) Feedback (§ \ref{['sec:fdb']}), and (d) Models (§ \ref{['sec:models']}).
  • Figure 2: A survey of Chain-of-X by taxonomies of nodes and tasks (only representative methods are listed due to space limitation and a more complete version can be found in Appendix \ref{['sec:app']}).
  • Figure 3: A simplified illustrative workflow of Chain-of-Experts xiao2024chainofexperts.
  • Figure 4: A Survey of Chain-of-X by Taxonomies of Nodes and Tasks.