Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Framework of Thoughts: A Foundation Framework for Dynamic and Optimized Reasoning based on Chains, Trees, and Graphs

Felix Fricke, Simon Malberg, Georg Groh

TL;DR

FoT presents a foundation framework for dynamic and optimized reasoning in LLMs, addressing the rigidity and inefficiency of existing prompting schemes by supporting evolving execution and reasoning graphs, safe parallelism, and persistent caching. By re-implementing Tree of Thoughts, Graph of Thoughts, and ProbTree within FoT, the authors demonstrate substantial speedups (average ~10.7x) and cost reductions, as well as improved task performance after hyperparameter and prompt optimization. The work contributes a practical toolkit for deploying adaptive reasoning schemes and provides open-source code to accelerate future research and development in dynamic reasoning for LLMs.

Abstract

Prompting schemes such as Chain of Thought, Tree of Thoughts, and Graph of Thoughts can significantly enhance the reasoning capabilities of large language models. However, most existing schemes require users to define static, problem-specific reasoning structures that lack adaptability to dynamic or unseen problem types. Additionally, these schemes are often under-optimized in terms of hyperparameters, prompts, runtime, and prompting cost. To address these limitations, we introduce Framework of Thoughts (FoT)--a general-purpose foundation framework for building and optimizing dynamic reasoning schemes. FoT comes with built-in features for hyperparameter tuning, prompt optimization, parallel execution, and intelligent caching, unlocking the latent performance potential of reasoning schemes. We demonstrate FoT's capabilities by implementing three popular schemes--Tree of Thoughts, Graph of Thoughts, and ProbTree--within FoT. We empirically show that FoT enables significantly faster execution, reduces costs, and achieves better task scores through optimization. We release our codebase to facilitate the development of future dynamic and efficient reasoning schemes.

Framework of Thoughts: A Foundation Framework for Dynamic and Optimized Reasoning based on Chains, Trees, and Graphs

TL;DR

FoT presents a foundation framework for dynamic and optimized reasoning in LLMs, addressing the rigidity and inefficiency of existing prompting schemes by supporting evolving execution and reasoning graphs, safe parallelism, and persistent caching. By re-implementing Tree of Thoughts, Graph of Thoughts, and ProbTree within FoT, the authors demonstrate substantial speedups (average ~10.7x) and cost reductions, as well as improved task performance after hyperparameter and prompt optimization. The work contributes a practical toolkit for deploying adaptive reasoning schemes and provides open-source code to accelerate future research and development in dynamic reasoning for LLMs.

Abstract

Prompting schemes such as Chain of Thought, Tree of Thoughts, and Graph of Thoughts can significantly enhance the reasoning capabilities of large language models. However, most existing schemes require users to define static, problem-specific reasoning structures that lack adaptability to dynamic or unseen problem types. Additionally, these schemes are often under-optimized in terms of hyperparameters, prompts, runtime, and prompting cost. To address these limitations, we introduce Framework of Thoughts (FoT)--a general-purpose foundation framework for building and optimizing dynamic reasoning schemes. FoT comes with built-in features for hyperparameter tuning, prompt optimization, parallel execution, and intelligent caching, unlocking the latent performance potential of reasoning schemes. We demonstrate FoT's capabilities by implementing three popular schemes--Tree of Thoughts, Graph of Thoughts, and ProbTree--within FoT. We empirically show that FoT enables significantly faster execution, reduces costs, and achieves better task scores through optimization. We release our codebase to facilitate the development of future dynamic and efficient reasoning schemes.
Paper Structure (33 sections, 3 equations, 5 figures, 6 tables)

This paper contains 33 sections, 3 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Overview of Prompting Schemes
  3. Framework of Thoughts
  4. Dynamic Graph Structures
  5. Safe Parallel Execution
  6. Efficient Caching
  7. Hyperparameter & Prompt Optimization
  8. Evaluation
  9. Results
  10. Conclusion
  11. Limitations & Future Work
  12. Reproducibility Statement
  13. Ethics Statement
  14. Appendix
  15. Illustrative Example
  16. ...and 18 more sections

Figures (5)

  • Figure 1: The execution graphs of a static prompting scheme implemented in the GoT framework versus a dynamic prompting scheme implemented in FoT. Nodes are operations, edges are information flows between them. While the graph structure is static and pre-planned in GoT, it can evolve dynamically in FoT (see steps 1-2). FoT executes operations in parallel (see step 2) and caches results of reoccurring operations (see step 3) to accelerate execution and reduce inference costs.
  • Figure 2: In the execution graph, nodes are operations and edges are connections that can carry thoughts. In the reasoning graph, nodes are thoughts and edges are dependencies. The execution graph can show the past, present, and future (i.e., completed, executing, and planned operations), whereas the reasoning graph only shows the past (thoughts produced by completed operations). The execution graph may be modified by operations, whereas the reasoning graph evolves as a byproduct.
  • Figure 3: Non-exhaustive list with three example operations. Operations can generate new thoughts and/or modify the execution graph.
  • Figure 4: Illustration of graph regions defined in Table \ref{['tab:subgraphs']}.
  • Figure 5: Exemplary evolution of the execution graph in an implementation of the Tree of Thoughts prompting scheme in Framework of Thoughts for the Game of 24 (see explanation in Section \ref{['sec:appendix-example']}).