Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Decision Making in Engineering System Design: A Deep Q-Learning Approach

Ramin Giahi, Cameron A. MacKenzie, Reyhaneh Bijari

TL;DR

This work targets dynamic engineering system design under evolving uncertainty by casting it as a multi‑stage stochastic problem and solving it with Deep Q‑learning. The approach learns policies directly from Monte Carlo simulations, avoiding the combinatorial burden of solving numerous optimization problems. Key contributions include detailing both Q‑learning and Deep Q‑learning frameworks for discrete, simulation-evaluated objectives and demonstrating their applicability on capacity‑expansion problems with price and demand uncertainties. The findings show that the Deep Q‑learning framework can closely reproduce analytically derived thresholds and achieve competitive profits, while significantly reducing computational effort, thereby offering a practical tool for decision-makers in complex, uncertain engineering contexts.

Abstract

Engineering system design, viewed as a decision-making process, faces challenges due to complexity and uncertainty. In this paper, we present a framework proposing the use of the Deep Q-learning algorithm to optimize the design of engineering systems. We outline a step-by-step framework for optimizing engineering system designs. The goal is to find policies that maximize the output of a simulation model given multiple sources of uncertainties. The proposed algorithm handles linear and non-linear multi-stage stochastic problems, where decision variables are discrete, and the objective function and constraints are assessed via a Monte Carlo simulation. We demonstrate the effectiveness of our proposed framework by solving two engineering system design problems in the presence of multiple uncertainties, such as price and demand.

Dynamic Decision Making in Engineering System Design: A Deep Q-Learning Approach

TL;DR

This work targets dynamic engineering system design under evolving uncertainty by casting it as a multi‑stage stochastic problem and solving it with Deep Q‑learning. The approach learns policies directly from Monte Carlo simulations, avoiding the combinatorial burden of solving numerous optimization problems. Key contributions include detailing both Q‑learning and Deep Q‑learning frameworks for discrete, simulation-evaluated objectives and demonstrating their applicability on capacity‑expansion problems with price and demand uncertainties. The findings show that the Deep Q‑learning framework can closely reproduce analytically derived thresholds and achieve competitive profits, while significantly reducing computational effort, thereby offering a practical tool for decision-makers in complex, uncertain engineering contexts.

Abstract

Engineering system design, viewed as a decision-making process, faces challenges due to complexity and uncertainty. In this paper, we present a framework proposing the use of the Deep Q-learning algorithm to optimize the design of engineering systems. We outline a step-by-step framework for optimizing engineering system designs. The goal is to find policies that maximize the output of a simulation model given multiple sources of uncertainties. The proposed algorithm handles linear and non-linear multi-stage stochastic problems, where decision variables are discrete, and the objective function and constraints are assessed via a Monte Carlo simulation. We demonstrate the effectiveness of our proposed framework by solving two engineering system design problems in the presence of multiple uncertainties, such as price and demand.
Paper Structure (11 sections, 10 equations, 6 figures, 1 table, 2 algorithms)

This paper contains 11 sections, 10 equations, 6 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Decision-Making Framework
  4. Q-learning
  5. Deep Q-learning
  6. Engineering System Design Illustrative Examples
  7. Capacity Expansion Problem with Price Uncertainty: Problem Formulation
  8. Capacity Expansion Problem with Price Uncertainty: Solution
  9. Capacity Expansion Problem with Price and Demand Uncertainty: Problem Formulation
  10. Capacity Expansion Problem with Price and Demand Uncertainty: Solution
  11. Conclusion

Figures (6)

  • Figure 1: Proposed framework for solving multi-stage stochastic problem
  • Figure 2: Optimal decision at stage two for two-stage problem with price uncertainty
  • Figure 3: Optimal decision at stages 2 and 3 for three-stages problem with price uncertainty
  • Figure 4: Distribution of uncertain parameters during the planning horizon
  • Figure 5: Three-stage problem with maximum capacity of two
  • ...and 1 more figures