Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Doc2AHP: Inferring Structured Multi-Criteria Decision Models via Semantic Trees with LLMs

Hongjia Wu, Shuai Zhou, Hongxin Zhang, Wei Chen

TL;DR

The paper tackles the problem of constructing structured, auditable multi-criteria decision models from unstructured documents. It introduces Doc2AHP, a two-phase framework that uses semantic geometry to generate an AHP hierarchy and a Leader-guided, multi-agent weighting scheme with consistency optimization, enabling reliable decision inference. The method demonstrates improved top-tier ranking quality (NDCG@5) and robust numerical consistency across model scales compared to baselines, particularly on semantically complex tasks. The results suggest a practical, interpretable path toward deploying rigorous MCDM systems on compute-constrained devices, by trading some speed for verifiable slow thinking and traceability.

Abstract

While Large Language Models (LLMs) demonstrate remarkable proficiency in semantic understanding, they often struggle to ensure structural consistency and reasoning reliability in complex decision-making tasks that demand rigorous logic. Although classical decision theories, such as the Analytic Hierarchy Process (AHP), offer systematic rational frameworks, their construction relies heavily on labor-intensive domain expertise, creating an "expert bottleneck" that hinders scalability in general scenarios. To bridge the gap between the generalization capabilities of LLMs and the rigor of decision theory, we propose Doc2AHP, a novel structured inference framework guided by AHP principles. Eliminating the need for extensive annotated data or manual intervention, our approach leverages the structural principles of AHP as constraints to direct the LLM in a constrained search within the unstructured document space, thereby enforcing the logical entailment between parent and child nodes. Furthermore, we introduce a multi-agent weighting mechanism coupled with an adaptive consistency optimization strategy to ensure the numerical consistency of weight allocation. Empirical results demonstrate that Doc2AHP not only empowers non-expert users to construct high-quality decision models from scratch but also significantly outperforms direct generative baselines in both logical completeness and downstream task accuracy.

Doc2AHP: Inferring Structured Multi-Criteria Decision Models via Semantic Trees with LLMs

TL;DR

The paper tackles the problem of constructing structured, auditable multi-criteria decision models from unstructured documents. It introduces Doc2AHP, a two-phase framework that uses semantic geometry to generate an AHP hierarchy and a Leader-guided, multi-agent weighting scheme with consistency optimization, enabling reliable decision inference. The method demonstrates improved top-tier ranking quality (NDCG@5) and robust numerical consistency across model scales compared to baselines, particularly on semantically complex tasks. The results suggest a practical, interpretable path toward deploying rigorous MCDM systems on compute-constrained devices, by trading some speed for verifiable slow thinking and traceability.

Abstract

While Large Language Models (LLMs) demonstrate remarkable proficiency in semantic understanding, they often struggle to ensure structural consistency and reasoning reliability in complex decision-making tasks that demand rigorous logic. Although classical decision theories, such as the Analytic Hierarchy Process (AHP), offer systematic rational frameworks, their construction relies heavily on labor-intensive domain expertise, creating an "expert bottleneck" that hinders scalability in general scenarios. To bridge the gap between the generalization capabilities of LLMs and the rigor of decision theory, we propose Doc2AHP, a novel structured inference framework guided by AHP principles. Eliminating the need for extensive annotated data or manual intervention, our approach leverages the structural principles of AHP as constraints to direct the LLM in a constrained search within the unstructured document space, thereby enforcing the logical entailment between parent and child nodes. Furthermore, we introduce a multi-agent weighting mechanism coupled with an adaptive consistency optimization strategy to ensure the numerical consistency of weight allocation. Empirical results demonstrate that Doc2AHP not only empowers non-expert users to construct high-quality decision models from scratch but also significantly outperforms direct generative baselines in both logical completeness and downstream task accuracy.
Paper Structure (19 sections, 7 equations, 1 figure, 4 tables, 1 algorithm)

This paper contains 19 sections, 7 equations, 1 figure, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Language Models for Complex Reasoning and Decision Support
  4. Automated Construction of AHP Models
  5. Methodology
  6. Overview of the Proposed Framework
  7. Probabilistic AHP Construction
  8. Decision Inference
  9. Document Embedding and Semantic Representation
  10. Semantic Hierarchical Clustering
  11. Recursive AHP Structure Generation
  12. Consistency-Aware Weight Estimation
  13. Decision Inference and Interpretability Support
  14. Experiments
  15. Experimental Setup
  16. ...and 4 more sections

Figures (1)

  • Figure 1: The pipeline of the Doc2AHP.