Diverge to Induce Prompting: Multi-Rationale Induction for Zero-Shot Reasoning

TL;DR

Diverge-to-Induce Prompting (DIP) tackles instability in zero-shot reasoning by first eliciting multiple diverse high-level rationales per question, elaborating each into a draft plan, and then inducing a final plan from these drafts before performing a single, final inference. Structured in three phases—Divergent Phase, Inductive Phase, and Inference Phase—DIP avoids repeated sampling or external voting modules while leveraging instance-level inductive reasoning. Empirical evaluation on BBH and LiveBench across six model families shows DIP consistently surpasses single-path prompting baselines (Z-CoT, S-CoT, R-CoT), with notable gains on challenging tasks and favorable cost-performance compared to self-consistency methods. The work highlights the practical value of multi-rationale planning for robust, efficient prompt-based reasoning in diverse models and tasks, while acknowledging computational overhead and areas for future generalization.

Abstract

To address the instability of unguided reasoning paths in standard Chain-of-Thought prompting, recent methods guide large language models (LLMs) by first eliciting a single reasoning strategy. However, relying on just one strategy for each question can still limit performance across diverse tasks. We propose Diverge-to-Induce Prompting (DIP), a framework that first prompts an LLM to generate multiple diverse high-level rationales for each question. Each rationale is then elaborated into a detailed, step-by-step draft plan. Finally, these draft plans are induced into a final plan. DIP enhances zero-shot reasoning accuracy without reliance on resource-intensive sampling. Experiments show that DIP outperforms single-strategy prompting, demonstrating the effectiveness of multi-plan induction for prompt-based reasoning.

Figures (14)

• Figure 1: Overview of the DIP framework. The process consists of three main phases: a Divergent Phase, where the model generates multiple high-level rationales and constructs a draft plan for each; an Inductive Phase, where all plans are induced into a final plan; and an Inference Phase, which produces the final reasoning and answer.
• Figure 2: BBH Salient Translation Error Detection example. Identify the error type in a German-to-English translation. Here, "Nattern" (colubrid family) is mistranslated as "Lantern family."
• Figure 3: Prompt template used for Rationale Generation in the DIP framework.
• Figure 4: Prompt template used for Draft Plan Generation in the DIP framework.
• Figure 5: Prompt template used for Draft Plan Induction in the DIP framework.