Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IAO Prompting: Making Knowledge Flow Explicit in LLMs through Structured Reasoning Templates

Aissatou Diallo, Antonis Bikakis, Luke Dickens, Anthony Hunter, Rob Miller

TL;DR

This work introduces IAO prompting, a structured template (Input-Action-Output with Subquestion) that makes how LLMs access and apply knowledge explicit during reasoning. By decomposing tasks into sequential steps with traceable knowledge inputs, planned actions, and produced outputs, IAO enables real-time verification, strengthens knowledge flow transparency, and helps identify gaps or misapplications. Across arithmetic, logical, commonsense, and symbolic reasoning tasks, IAO improves zero-shot performance and maintains competitive results against planning-based baselines, while enabling better interpretability through its step-by-step structure. Human evaluators also find IAO reasoning chains more transparent and useful for error detection, suggesting practical benefits for reliable knowledge application in real-world AI systems. The approach shows domain independence and supports extensions such as two-stage prompting and few-shot demonstrations, with considerations for output length and bias mitigation in future work.

Abstract

While Large Language Models (LLMs) demonstrate impressive reasoning capabilities, understanding and validating their knowledge utilization remains challenging. Chain-of-thought (CoT) prompting partially addresses this by revealing intermediate reasoning steps, but the knowledge flow and application remain implicit. We introduce IAO (Input-Action-Output) prompting, a structured template-based method that explicitly models how LLMs access and apply their knowledge during complex reasoning tasks. IAO decomposes problems into sequential steps, each clearly identifying the input knowledge being used, the action being performed, and the resulting output. This structured decomposition enables us to trace knowledge flow, verify factual consistency, and identify potential knowledge gaps or misapplications. Through experiments across diverse reasoning tasks, we demonstrate that IAO not only improves zero-shot performance but also provides transparency in how LLMs leverage their stored knowledge. Human evaluation confirms that this structured approach enhances our ability to verify knowledge utilization and detect potential hallucinations or reasoning errors. Our findings provide insights into both knowledge representation within LLMs and methods for more reliable knowledge application.

IAO Prompting: Making Knowledge Flow Explicit in LLMs through Structured Reasoning Templates

TL;DR

This work introduces IAO prompting, a structured template (Input-Action-Output with Subquestion) that makes how LLMs access and apply knowledge explicit during reasoning. By decomposing tasks into sequential steps with traceable knowledge inputs, planned actions, and produced outputs, IAO enables real-time verification, strengthens knowledge flow transparency, and helps identify gaps or misapplications. Across arithmetic, logical, commonsense, and symbolic reasoning tasks, IAO improves zero-shot performance and maintains competitive results against planning-based baselines, while enabling better interpretability through its step-by-step structure. Human evaluators also find IAO reasoning chains more transparent and useful for error detection, suggesting practical benefits for reliable knowledge application in real-world AI systems. The approach shows domain independence and supports extensions such as two-stage prompting and few-shot demonstrations, with considerations for output length and bias mitigation in future work.

Abstract

While Large Language Models (LLMs) demonstrate impressive reasoning capabilities, understanding and validating their knowledge utilization remains challenging. Chain-of-thought (CoT) prompting partially addresses this by revealing intermediate reasoning steps, but the knowledge flow and application remain implicit. We introduce IAO (Input-Action-Output) prompting, a structured template-based method that explicitly models how LLMs access and apply their knowledge during complex reasoning tasks. IAO decomposes problems into sequential steps, each clearly identifying the input knowledge being used, the action being performed, and the resulting output. This structured decomposition enables us to trace knowledge flow, verify factual consistency, and identify potential knowledge gaps or misapplications. Through experiments across diverse reasoning tasks, we demonstrate that IAO not only improves zero-shot performance but also provides transparency in how LLMs leverage their stored knowledge. Human evaluation confirms that this structured approach enhances our ability to verify knowledge utilization and detect potential hallucinations or reasoning errors. Our findings provide insights into both knowledge representation within LLMs and methods for more reliable knowledge application.

Paper Structure

This paper contains 59 sections, 14 equations, 2 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Prompting
  4. Problem Decomposition in LLMs
  5. Knowledge Verification in LLMs
  6. IAO Prompting
  7. Experimental Setup
  8. Tasks
  9. Models
  10. Baselines
  11. Results
  12. Arithmetic Reasoning
  13. GPT-4
  14. PALM-2
  15. Logical Reasoning
  16. ...and 44 more sections

Figures (2)

  • Figure 1: Illustration of IAO prompting demonstrating how knowledge is structured and applied through explicit Input-Action-Output steps. Each step's output becomes verified knowledge for subsequent reasoning.
  • Figure 2: Comparison of knowledge application between IAO prompting and zero-shot CoT using PALM-2 on GSM8k. IAO's structured format reveals how knowledge is accessed and applied at each step, while CoT misses crucial information.