ARGORA: Orchestrated Argumentation for Causally Grounded LLM Reasoning and Decision Making
Youngjin Jin, Hanna Kim, Kwanwoo Kim, Chanhee Lee, Seungwon Shin
TL;DR
ARGORA introduces an orchestrated argumentation framework that converts multi-expert LLM deliberations into explicit Quantitative Bipolar Argumentation Frameworks (QBAFs) and treats their evaluation as a structural causal model (SCM). This enables edge-local counterfactual interventions to diagnose which arguments drive decisions and why, and adds an observation-aligned override mechanism to reconcile internal consensus with external judgments. Across diverse benchmarks, ARGORA delivers competitive accuracy and demonstrates a corrective tendency when experts disagree, while providing mechanistic explanations and robust diagnostics of decisive arguments. The approach advances explainability and reliability in multi-LLM reasoning, offering a principled foundation for analyzing and improving complex, high-stakes decision making.
Abstract
Existing multi-expert LLM systems gather diverse perspectives but combine them through simple aggregation, obscuring which arguments drove the final decision. We introduce ARGORA, a framework that organizes multi-expert discussions into explicit argumentation graphs showing which arguments support or attack each other. By casting these graphs as causal models, ARGORA can systematically remove individual arguments and recompute outcomes, identifying which reasoning chains were necessary and whether decisions would change under targeted modifications. We further introduce a correction mechanism that aligns internal reasoning with external judgments when they disagree. Across diverse benchmarks and an open-ended use case, ARGORA achieves competitive accuracy and demonstrates corrective behavior: when experts initially disagree, the framework resolves disputes toward correct answers more often than it introduces new errors, while providing causal diagnostics of decisive arguments.