Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An efficient mixed-integer linear programming formulation for solving influence diagrams

Topias Terho, Fabricio Oliveira, Ahti Salo, Pedro Munari

TL;DR

The paper tackles solving influence diagrams with MILP when prior RJT and Decision Programming approaches struggle, especially for nonperiodic structures and large-state nodes. It introduces an observation-based reformulation (DP-R2) that aggregates paths by observable segments, replacing exponential path variables with compact, aggregated ones while preserving optimality. It also extends the framework to risk-aware objectives (CVaR) and chance constraints, enabling robust decision-making under uncertainty. Through computational experiments on turbine, oil wildcatter, and water management problems, the DP-R2 formulation demonstrates substantial speedups and the ability to solve larger, more complex diagrams that were previously intractable, complementing existing RJT methods. Overall, the approach significantly broadens the practical applicability of MILP-based influence diagram solutions, with clear directions for further improvement via decomposition and stronger inequality formulations.

Abstract

Influence diagrams represent decision-making problems with interdependencies between random events, decisions, and consequences. Traditionally, they have been solved using algorithms that determine the expected utility-maximizing decision strategy. In contrast, state-of-the-art solution approaches convert influence diagrams into a mixed-integer linear programming (MILP) model, which can be solved with powerful off-the-shelf MILP solvers. From a computational standpoint, the existing MILP formulations can be efficiently solved when applied to influence diagrams that represent periodic (or sequential) decision processes, which can be cast as partially observable Markov Decision Processes. However, they are inefficient in problems that lack a periodic structure or if the nodes in the influence diagram have large state spaces, thus limiting their practical use. In this paper, we present an efficient MILP formulation that is specifically designed for influence diagrams that are challenging for the earlier MILP formulation-based methods. Additionally, we present how the proposed formulation can be adapted to maximize conditional value-at-risk and how chance and logical constraints can be incorporated into the formulation, thus retaining the modeling flexibility of the MILP-based methods. Finally, we perform computational experiments addressing problems from the literature and compare the computational efficiency of the proposed formulation against the available MILP formulations for the reported influence diagrams. We find that the MILP models based on the proposed formulations can be solved significantly more efficiently compared to the state-of-the-art when solving influence diagrams that cannot be cast as partially observable Markov decision processes.

An efficient mixed-integer linear programming formulation for solving influence diagrams

TL;DR

The paper tackles solving influence diagrams with MILP when prior RJT and Decision Programming approaches struggle, especially for nonperiodic structures and large-state nodes. It introduces an observation-based reformulation (DP-R2) that aggregates paths by observable segments, replacing exponential path variables with compact, aggregated ones while preserving optimality. It also extends the framework to risk-aware objectives (CVaR) and chance constraints, enabling robust decision-making under uncertainty. Through computational experiments on turbine, oil wildcatter, and water management problems, the DP-R2 formulation demonstrates substantial speedups and the ability to solve larger, more complex diagrams that were previously intractable, complementing existing RJT methods. Overall, the approach significantly broadens the practical applicability of MILP-based influence diagram solutions, with clear directions for further improvement via decomposition and stronger inequality formulations.

Abstract

Influence diagrams represent decision-making problems with interdependencies between random events, decisions, and consequences. Traditionally, they have been solved using algorithms that determine the expected utility-maximizing decision strategy. In contrast, state-of-the-art solution approaches convert influence diagrams into a mixed-integer linear programming (MILP) model, which can be solved with powerful off-the-shelf MILP solvers. From a computational standpoint, the existing MILP formulations can be efficiently solved when applied to influence diagrams that represent periodic (or sequential) decision processes, which can be cast as partially observable Markov Decision Processes. However, they are inefficient in problems that lack a periodic structure or if the nodes in the influence diagram have large state spaces, thus limiting their practical use. In this paper, we present an efficient MILP formulation that is specifically designed for influence diagrams that are challenging for the earlier MILP formulation-based methods. Additionally, we present how the proposed formulation can be adapted to maximize conditional value-at-risk and how chance and logical constraints can be incorporated into the formulation, thus retaining the modeling flexibility of the MILP-based methods. Finally, we perform computational experiments addressing problems from the literature and compare the computational efficiency of the proposed formulation against the available MILP formulations for the reported influence diagrams. We find that the MILP models based on the proposed formulations can be solved significantly more efficiently compared to the state-of-the-art when solving influence diagrams that cannot be cast as partially observable Markov decision processes.
Paper Structure (19 sections, 6 theorems, 19 equations, 4 figures, 8 tables)

This paper contains 19 sections, 6 theorems, 19 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Influence Diagrams
  4. Decision Programming
  5. Contributions
  6. Reformulated Decision Programming model
  7. Alternative objective functions and risk constraints
  8. Conditional value-at-risk
  9. Chance constraints
  10. Computational considerations
  11. Computational Experiments
  12. Assessing the optimization models
  13. Turbine inspection and maintenance problem
  14. Extended oil wildcatter problem
  15. Risk-averse water management problem
  16. ...and 4 more sections

Key Result

Proposition 1

Let $(x,z)$ be a feasible solution for the constraint set eq:dp_strategy1-eq:dp_path_compat. Let $x^{*}$ be such that $x^{*}(s) = 1$ if $\max_{s' \in E^{>}(s_O)}x(s') > 0$ and $x^{*}(s) = 0$ otherwise. Then, $(x^{*},z)$ is feasible for the constraint set eq:dp_strategy1-eq:dp_path_compat. Moreover,

Figures (4)

  • Figure 1: Influence diagram of the turbine inspection and maintenance problem
  • Figure 2: Influence diagram of the extended oil wildcatter problem, where $M=2$
  • Figure 3: Influence diagram of the risk-averse water resource management problem
  • Figure 4: Average relative improvement of optimal decision strategies solved by using a discretization of specific size when compared to a model solved when using 2 discrete states.

Theorems & Definitions (12)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • Proposition 5
  • proof
  • ...and 2 more