Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Tree Generation for Globally Optimal Decisions under Probabilistic Outcomes

Berk Ozturk, She'ifa Punla-Green, Les Servi

TL;DR

The paper tackles the problem of generating policies for sequential, uncertain decision problems with interdependent actions by constructing globally optimal decision trees. It introduces a methodology that combines dynamic programming with mixed-integer linear optimization to produce trees that maximize expected rewards while pruning vast portions of the state space through reward-driven pruning and dominance analysis. The approach comprises three stages: full graph generation with pruning, reduced graph construction via a Phi score recursion, and a tree selection step that optionally uses a secondary objective to minimize tree size; an LP formulation is also offered for validation and adversarial extensions. Computational results on randomized graphs demonstrate linear scaling in the number of explored states and substantial speed-ups over naive generation, highlighting the method’s practical scalability for large, structured problems. The work has potential impact across healthcare, wargaming, and cybersecurity by enabling globally optimal, interpretable decision policies under uncertainty without extensive parameter tuning, and it points toward future adversarial extensions and further state-space reduction techniques.

Abstract

Many real-world problems require making sequences of decisions where the outcomes of each decision are probabilistic and uncertain, and the availability of different actions is constrained by the outcomes of previous actions. There is a need to generate policies that are adaptive to uncertainty, globally optimal, and yet scalable as the state space grows. In this paper, we propose the generation of optimal decision trees, which dictate which actions should be implemented in different outcome scenarios, while maximizing the expected reward of the strategy. Using a combination of dynamic programming and mixed-integer linear optimization, the proposed methods scale to problems with large but finite state spaces, using problem-specific information to prune away large subsets of the state space that do not yield progress towards rewards. We demonstrate that the presented approach is able to find the globally optimal decision tree in linear time with respect to the number states explored.

Efficient Tree Generation for Globally Optimal Decisions under Probabilistic Outcomes

TL;DR

The paper tackles the problem of generating policies for sequential, uncertain decision problems with interdependent actions by constructing globally optimal decision trees. It introduces a methodology that combines dynamic programming with mixed-integer linear optimization to produce trees that maximize expected rewards while pruning vast portions of the state space through reward-driven pruning and dominance analysis. The approach comprises three stages: full graph generation with pruning, reduced graph construction via a Phi score recursion, and a tree selection step that optionally uses a secondary objective to minimize tree size; an LP formulation is also offered for validation and adversarial extensions. Computational results on randomized graphs demonstrate linear scaling in the number of explored states and substantial speed-ups over naive generation, highlighting the method’s practical scalability for large, structured problems. The work has potential impact across healthcare, wargaming, and cybersecurity by enabling globally optimal, interpretable decision policies under uncertainty without extensive parameter tuning, and it points toward future adversarial extensions and further state-space reduction techniques.

Abstract

Many real-world problems require making sequences of decisions where the outcomes of each decision are probabilistic and uncertain, and the availability of different actions is constrained by the outcomes of previous actions. There is a need to generate policies that are adaptive to uncertainty, globally optimal, and yet scalable as the state space grows. In this paper, we propose the generation of optimal decision trees, which dictate which actions should be implemented in different outcome scenarios, while maximizing the expected reward of the strategy. Using a combination of dynamic programming and mixed-integer linear optimization, the proposed methods scale to problems with large but finite state spaces, using problem-specific information to prune away large subsets of the state space that do not yield progress towards rewards. We demonstrate that the presented approach is able to find the globally optimal decision tree in linear time with respect to the number states explored.

Paper Structure

This paper contains 18 sections, 15 equations, 5 figures, 3 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Decision trees
  4. Sequential decision making under uncertainty
  5. Applications of decision trees
  6. Illustrative example
  7. Methodology
  8. Full graph generation
  9. Accelerating full graph generation
  10. Reduced graph generation
  11. Alternative approach: linear programming
  12. Tree selection
  13. Computational Results
  14. Conclusion
  15. Acknowledgments
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Example optimal decision tree with binary splits, where each node is a state with an action prescription.
  • Figure 2: Action dependency graph for illustrative example.
  • Figure 3: Optimal decision tree for illustrative example, budget = 6.
  • Figure 4: Example random action dependency graph with 20 actions, and OR and AND prerequisite relationships.
  • Figure 5: The optimal method scales linearly with the number of states explored.