Structuring Reasoning for Complex Rules Beyond Flat Representations

TL;DR

The paper tackles the challenge of reasoning over densely interdependent rule systems, where traditional prompting strategies fail due to flat representations and error propagation. It introduces Dynamic Adjudication Template (DAT), a three-stage, template-guided reasoning framework with a dynamic template library and adaptive template selection that enforces structured, verifiable rule-based inference. Empirical results on the EVADE benchmark show that DAT substantially improves accuracy, allowing smaller models to rival or surpass much larger models while maintaining efficiency. The work further demonstrates the approach's extensibility to vision-language models and highlights the practical significance for high-stakes, rule-intensive domains like e-commerce, law, and finance.

Abstract

Large language models (LLMs) face significant challenges when processing complex rule systems, as they typically treat interdependent rules as unstructured textual data rather than as logically organized frameworks. This limitation results in reasoning divergence, where models often overlook critical rule dependencies essential for accurate interpretation. Although existing approaches such as Chain-of-Thought (CoT) reasoning have shown promise, they lack systematic methodologies for structured rule processing and are particularly susceptible to error propagation through sequential reasoning chains. To address these limitations, we propose the Dynamic Adjudication Template (DAT), a novel framework inspired by expert human reasoning processes. DAT structures the inference mechanism into three methodical stages: qualitative analysis, evidence gathering, and adjudication. During the qualitative analysis phase, the model comprehensively evaluates the contextual landscape. The subsequent evidence gathering phase involves the targeted extraction of pertinent information based on predefined template elements ([placeholder]), followed by systematic verification against applicable rules. Finally, in the adjudication phase, the model synthesizes these validated components to formulate a comprehensive judgment. Empirical results demonstrate that DAT consistently outperforms conventional CoT approaches in complex rule-based tasks. Notably, DAT enables smaller language models to match, and in some cases exceed, the performance of significantly larger LLMs, highlighting its efficiency and effectiveness in managing intricate rule systems.

Figures (14)

• Figure 1: Illustration of different reasoning process. Dynamic Adjudication Template enables large language models to organizes the inference mechanism into a structured three-stage process.
• Figure 2: (Left) The architecture of the Global-Local Selector. (Right) When processing complex rule systems, LLM utilizes the selected DAT through Qualitative Analysis, Evidence Gathering, and Adjudication to generate a comprehensive judgment.
• Figure 3: Ablation analysis of partial accuracy for Qwen-2.5-7B and InternLM3-8B. Baseline uses no enhancements; "*" adds evidence gathering; "#" adds adjudication on top.
• Figure 4: Case study contrasting DAT’s structured reasoning flow with CoT’s flat response, showing how DAT aligns rules and evidence to reach a consistent decision.
• Figure 5: Examples showcasing the effectiveness of DAT across six tasks.