Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-optimization in distributed manufacturing systems using Modular State-based Stackelberg Games

Steve Yuwono, Ahmar Kamal Hussain, Dorothea Schwung, Andreas Schwung

Abstract

In this study, we introduce Modular State-based Stackelberg Games (Mod-SbSG), a novel game structure developed for distributed self-learning in modular manufacturing systems. Mod-SbSG enhances cooperative decision-making among self-learning agents within production systems by integrating State-based Potential Games (SbPG) with Stackelberg games. This hierarchical structure assigns more important modules of the manufacturing system a first-mover advantage, while less important modules respond optimally to the leaders' decisions. This decision-making process differs from typical multi-agent learning algorithms in manufacturing systems, where decisions are made simultaneously. We provide convergence guarantees for the novel game structure and design learning algorithms to account for the hierarchical game structure. We further analyse the effects of single-leader/multiple-follower and multiple-leader/multiple-follower scenarios within a Mod-SbSG. To assess its effectiveness, we implement and test Mod-SbSG in an industrial control setting using two laboratory-scale testbeds featuring sequential and serial-parallel processes. The proposed approach delivers promising results compared to the vanilla SbPG, which reduces overflow by 97.1%, and in some cases, prevents overflow entirely. Additionally, it decreases power consumption by 5-13% while satisfying the production demand, which significantly improves potential (global objective) values.

Self-optimization in distributed manufacturing systems using Modular State-based Stackelberg Games

Abstract

In this study, we introduce Modular State-based Stackelberg Games (Mod-SbSG), a novel game structure developed for distributed self-learning in modular manufacturing systems. Mod-SbSG enhances cooperative decision-making among self-learning agents within production systems by integrating State-based Potential Games (SbPG) with Stackelberg games. This hierarchical structure assigns more important modules of the manufacturing system a first-mover advantage, while less important modules respond optimally to the leaders' decisions. This decision-making process differs from typical multi-agent learning algorithms in manufacturing systems, where decisions are made simultaneously. We provide convergence guarantees for the novel game structure and design learning algorithms to account for the hierarchical game structure. We further analyse the effects of single-leader/multiple-follower and multiple-leader/multiple-follower scenarios within a Mod-SbSG. To assess its effectiveness, we implement and test Mod-SbSG in an industrial control setting using two laboratory-scale testbeds featuring sequential and serial-parallel processes. The proposed approach delivers promising results compared to the vanilla SbPG, which reduces overflow by 97.1%, and in some cases, prevents overflow entirely. Additionally, it decreases power consumption by 5-13% while satisfying the production demand, which significantly improves potential (global objective) values.

Paper Structure

This paper contains 31 sections, 3 theorems, 21 equations, 13 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature review
  3. Self-learning manufacturing systems using artificial intelligence
  4. Stackelberg games for engineering applications
  5. Problem description
  6. Modular State-based Stackelberg Games
  7. SbPG for leader and follower groups
  8. Leader-follower game as a Stackelberg game
  9. Overall game structure
  10. Convergence analysis
  11. Learning dynamics
  12. Policy representation using performance maps
  13. Learning update rule
  14. Approximation of gradient descent
  15. Multi-step updates for followers
  16. ...and 16 more sections

Key Result

Theorem 1

Given Assumptions 1-4 from Schwung2022, the cooperative game between leaders, $\Gamma_L(\mathcal{L}, \mathcal{A}_L, \{U_l\}, {S}, P, \phi_L)$, as defined in Def. def:sbpg_leader constitute an SbPG.

Figures (13)

  • Figure 1: An illustration of modular production units Yuwono2024b.
  • Figure 2: A schematic diagram of a production chain featuring serial-parallel connected subsystems.
  • Figure 3: An overview of Mod-SbSG in distributed manufacturing systems.
  • Figure 4: 5 x 5 performance map representation within a 2D state space in SbPG Yuwono2024a.
  • Figure 5: The updated structure of performance maps for the followers in Mod-SbSG on SbPG.
  • ...and 8 more figures

Theorems & Definitions (11)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Theorem 1
  • Proof
  • Theorem 2
  • Proof
  • Definition 5
  • Theorem 3
  • ...and 1 more