Understanding When Tree of Thoughts Succeeds: Larger Models Excel in Generation, Not Discrimination

TL;DR

It is found that the generator plays a more critical role than the discriminator in driving the success of ToT, and models across different scales exhibit comparable discrimination capabilities, yet differ significantly in their generative performance for ToT.

Abstract

Tree of Thoughts (ToT) is a reasoning strategy for Large Language Models (LLMs) that employs a generator to suggest reasoning steps and a discriminator to decide which steps to implement. ToT demonstrates strong performance on reasoning tasks, often surpassing simple methods such as Input-Output (IO) prompting and Chain-of-Thought (CoT) reasoning. However, ToT does not consistently outperform such simpler methods across all models, leaving large knowledge gaps on the conditions under which ToT is most beneficial. In this paper, we analyze the roles of the generator and discriminator separately to better understand the conditions when ToT is beneficial. We find that the generator plays a more critical role than the discriminator in driving the success of ToT. Scaling the generator leads to notable improvements in ToT performance, even when using a smaller model as the discriminator, whereas scaling the discriminator with a fixed generator yields only marginal gains. Our results show that models across different scales exhibit comparable discrimination capabilities, yet differ significantly in their generative performance for ToT.

Figures (7)

• Figure 1: Core Mechanism of ToT. It employs a generator to suggest intermediate steps and a discriminator to decide which steps to take.
• Figure 2: Diagram of relationships among different types of steps in ToT.
• Figure 3: Illustration of Game of 24 and Knights and Knaves under ToT setting. The generator proposes possible intermediate steps, which will be evaluated by the discriminator. We denote the viable/inviable intermediate partial solutions in green/red.
• Figure 4: Illustration of IO, CoT and ToT.
• Figure 5: Impact of different models as generators on the overall performance of ToT against oracle discriminators. The lines plot illustrates the average success rate when paired with oracle discriminators. We also plot the performance of IO, CoT, and in combination with a random discriminator.