Self-guided Knowledgeable Network of Thoughts: Amplifying Reasoning with Large Language Models

TL;DR

The paper presents Knowledgeable Network of Thoughts (kNoT), a self-guided prompting framework that lets LLMs draft and execute executable reasoning networks via a structured LLM Workflow Template (LWT). By decoupling plan generation from execution and enabling arbitrary networked reasoning with elementary operations, kNoT reduces human prompt engineering while improving robustness and scalability across natural language, symbolic, and arithmetic tasks. Empirical results across six use cases show significant performance gains over CoT, ToT, and GoT, with lower prompting costs and reduced task-specific labor. This approach advances intelligence amplification in LLMs by enabling autonomous, task-specific network construction and execution within a self-contained prompt framework.

Abstract

We introduce Knowledgeable Network of Thoughts (kNoT): a prompt scheme that advances the capabilities of large language models (LLMs) beyond existing paradigms like Chain-of-Thought (CoT), Tree of Thoughts (ToT), and Graph of Thoughts (GoT). The key innovation of kNoT is the LLM Workflow Template (LWT), which allows for an executable plan to be specified by LLMs for LLMs. LWT allows these plans to be arbitrary networks, where single-step LLM operations are nodes, and edges correspond to message passing between these steps. Furthermore, LWT supports selection of individual elements through indexing, facilitating kNoT to produce intricate plans where each LLM operation can be limited to elementary operations, greatly enhancing reliability over extended task sequences. We demonstrate that kNoT significantly outperforms the state of the art on six use cases, while reducing the need for extensive prompt engineering. For instance, kNoT finds 92% accuracy for sorting 32 numbers over 12% and 31% for ToT and GoT, while utilizing up to 84.4% and 87.3% less task-specific prompts, respectively.

Figures (4)

• Figure 1: Comparison of Knowledgeable Network of Thoughts (kNoT) to other prompting strategies.
• Figure 2: Illustrations of human labor (orange block) and LLM operation (blue block) required for the prompt solution procedure. Gray stripes indicate the labor involved in designing constant prompts, which do not require redesign.
• Figure 3: Illustration of the Self-guided Knowledgeable Network of Thoughts (kNoT). The prompts in kNoT include constant prompts and task-specific prompts, comprising a context description and a LWT example. kNoT first generates a LLM solution plan from the LLM based on the input query. Then, it transforms the LLM solution plan into an LWT-formatted script. Finally, the bottom-right diagram visualizes the execution of LWT-formatted script, forming a network of thought processes, where straight or curved arrows full or list-indexed message passing. Throughout the illustration, $\Rightarrow$ indicates LLM operations.
• Figure 4: Scalability analysis across varying lengths of addition sequences.

Theorems & Definitions (3)

• Definition 1: Prompt Engineering Problem
• Definition 2: Prompt Solution Procedure
• Definition 3: LLM Workflow Template (LWT)