Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Teaching LLMs to Ask: Self-Querying Category-Theoretic Planning for Under-Specified Reasoning

Shuhui Qu

TL;DR

SQ-BCP introduces a principled framework for LLM-based planning under partial observability by explicitly modeling preconditions with Sat/Viol/Unk labels, and resolving unknowns through targeted self-querying and bridging actions. It integrates a bidirectional search with a pullback-based categorical verifier, while using a distance-based score solely for ranking and pruning. The approach achieves substantially lower resource-violation rates on WikiHow and RecipeNLG compared with strong baselines, without sacrificing plan quality, and its theoretical analysis guarantees termination and correctness under bounded assumptions. This work advances executable planning for natural-language tasks in open-world contexts, where critical facts are latent or withheld in prompts, and demonstrates practical improvements in feasibility that are relevant for real-world instruction following and task execution.

Abstract

Inference-time planning with large language models frequently breaks under partial observability: when task-critical preconditions are not specified at query time, models tend to hallucinate missing facts or produce plans that violate hard constraints. We introduce \textbf{Self-Querying Bidirectional Categorical Planning (SQ-BCP)}, which explicitly represents precondition status (\texttt{Sat}/\texttt{Viol}/\texttt{Unk}) and resolves unknowns via (i) targeted self-queries to an oracle/user or (ii) \emph{bridging} hypotheses that establish the missing condition through an additional action. SQ-BCP performs bidirectional search and invokes a pullback-based verifier as a categorical certificate of goal compatibility, while using distance-based scores only for ranking and pruning. We prove that when the verifier succeeds and hard constraints pass deterministic checks, accepted plans are compatible with goal requirements; under bounded branching and finite resolution depth, SQ-BCP finds an accepting plan when one exists. Across WikiHow and RecipeNLG tasks with withheld preconditions, SQ-BCP reduces resource-violation rates to \textbf{14.9\%} and \textbf{5.8\%} (vs.\ \textbf{26.0\%} and \textbf{15.7\%} for the best baseline), while maintaining competitive reference quality.

Teaching LLMs to Ask: Self-Querying Category-Theoretic Planning for Under-Specified Reasoning

TL;DR

SQ-BCP introduces a principled framework for LLM-based planning under partial observability by explicitly modeling preconditions with Sat/Viol/Unk labels, and resolving unknowns through targeted self-querying and bridging actions. It integrates a bidirectional search with a pullback-based categorical verifier, while using a distance-based score solely for ranking and pruning. The approach achieves substantially lower resource-violation rates on WikiHow and RecipeNLG compared with strong baselines, without sacrificing plan quality, and its theoretical analysis guarantees termination and correctness under bounded assumptions. This work advances executable planning for natural-language tasks in open-world contexts, where critical facts are latent or withheld in prompts, and demonstrates practical improvements in feasibility that are relevant for real-world instruction following and task execution.

Abstract

Inference-time planning with large language models frequently breaks under partial observability: when task-critical preconditions are not specified at query time, models tend to hallucinate missing facts or produce plans that violate hard constraints. We introduce \textbf{Self-Querying Bidirectional Categorical Planning (SQ-BCP)}, which explicitly represents precondition status (\texttt{Sat}/\texttt{Viol}/\texttt{Unk}) and resolves unknowns via (i) targeted self-queries to an oracle/user or (ii) \emph{bridging} hypotheses that establish the missing condition through an additional action. SQ-BCP performs bidirectional search and invokes a pullback-based verifier as a categorical certificate of goal compatibility, while using distance-based scores only for ranking and pruning. We prove that when the verifier succeeds and hard constraints pass deterministic checks, accepted plans are compatible with goal requirements; under bounded branching and finite resolution depth, SQ-BCP finds an accepting plan when one exists. Across WikiHow and RecipeNLG tasks with withheld preconditions, SQ-BCP reduces resource-violation rates to \textbf{14.9\%} and \textbf{5.8\%} (vs.\ \textbf{26.0\%} and \textbf{15.7\%} for the best baseline), while maintaining competitive reference quality.
Paper Structure (58 sections, 3 theorems, 55 equations, 2 figures, 3 tables)

This paper contains 58 sections, 3 theorems, 55 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM Planning.
  4. Information Seeking and Question Generation.
  5. Classical Planning and Preconditions.
  6. Neuro-Symbolic and Compositional Reasoning.
  7. Verification and Constraint Satisfaction.
  8. Planning Under Uncertainty and Interaction.
  9. Methodology
  10. Problem Formulation
  11. Heuristic distance for ranking.
  12. Partial observability and hypothesis chains.
  13. State Space and Planning Category
  14. Hypothesis Structure
  15. Precondition labels.
  16. ...and 43 more sections

Key Result

Theorem 4.1

For any hypothesis $h$ with finite unknowns $U(w,h)$, the refinement procedure (Section sec:refine) terminates after at most $|U(w,h)| \cdot (T_{\text{bridge}} + 1)$ steps, producing a hypothesis with all preconditions classified as Sat or Viol.

Figures (2)

  • Figure 1: Bidirectional search from $w_0$ and $w^*$ with SQ-BCP self-querying and bridging hypotheses to resolve missing preconditions before categorical verification at meet points.
  • Figure 2: Query and hypothesis counts vs. missing preconditions. As uncertainty increases, SQ-BCP generates more queries (blue) and explores more hypotheses (green) to resolve unknowns.

Theorems & Definitions (9)

  • Definition 3.1: Planning Problem
  • Theorem 4.1: Refinement Terminates
  • proof
  • Theorem 4.2: Verification Certifies Correctness
  • proof
  • Remark 4.3
  • Theorem 4.4: Bounded Search Complexity
  • proof
  • Remark 4.5