Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automated decision-making for dynamic task assignment at scale

Riccardo Lo Bianco, Willem van Jaarsveld, Jeroen Middelhuis, Luca Begnardi, Remco Dijkman

TL;DR

The paper tackles a real-world DTAP where cases unfold as stochastic sequences of activities and timely resource assignments are needed to minimize average cycle time. It introduces a DRL-based Decision Support System with a graph-based observation structure and an assignment-node action mechanism, backed by a reward design proven to equate to the cycle-time objective via Little's Law. The approach, built on PPO and GNNs, demonstrates strong performance relative to baselines across five real-world DTAP instances and shows robust generalization to longer horizons and different datasets. This work advances scalable, transferable decision policies for complex, real-time task assignment in operational settings. It also outlines practical limitations and directions for extending realism (e.g., non-stationary dynamics and parallel activities).

Abstract

The Dynamic Task Assignment Problem (DTAP) concerns matching resources to tasks in real time while minimizing some objectives, like resource costs or task cycle time. In this work, we consider a DTAP variant where every task is a case composed of a stochastic sequence of activities. The DTAP, in this case, involves the decision of which employee to assign to which activity to process requests as quickly as possible. In recent years, Deep Reinforcement Learning (DRL) has emerged as a promising tool for tackling this DTAP variant, but most research is limited to solving small-scale, synthetic problems, neglecting the challenges posed by real-world use cases. To bridge this gap, this work proposes a DRL-based Decision Support System (DSS) for real-world scale DTAPS. To this end, we introduce a DRL agent with two novel elements: a graph structure for observations and actions that can effectively represent any DTAP and a reward function that is provably equivalent to the objective of minimizing the average cycle time of tasks. The combination of these two novelties allows the agent to learn effective and generalizable assignment policies for real-world scale DTAPs. The proposed DSS is evaluated on five DTAP instances whose parameters are extracted from real-world logs through process mining. The experimental evaluation shows how the proposed DRL agent matches or outperforms the best baseline in all DTAP instances and generalizes on different time horizons and across instances.

Automated decision-making for dynamic task assignment at scale

TL;DR

The paper tackles a real-world DTAP where cases unfold as stochastic sequences of activities and timely resource assignments are needed to minimize average cycle time. It introduces a DRL-based Decision Support System with a graph-based observation structure and an assignment-node action mechanism, backed by a reward design proven to equate to the cycle-time objective via Little's Law. The approach, built on PPO and GNNs, demonstrates strong performance relative to baselines across five real-world DTAP instances and shows robust generalization to longer horizons and different datasets. This work advances scalable, transferable decision policies for complex, real-time task assignment in operational settings. It also outlines practical limitations and directions for extending realism (e.g., non-stationary dynamics and parallel activities).

Abstract

The Dynamic Task Assignment Problem (DTAP) concerns matching resources to tasks in real time while minimizing some objectives, like resource costs or task cycle time. In this work, we consider a DTAP variant where every task is a case composed of a stochastic sequence of activities. The DTAP, in this case, involves the decision of which employee to assign to which activity to process requests as quickly as possible. In recent years, Deep Reinforcement Learning (DRL) has emerged as a promising tool for tackling this DTAP variant, but most research is limited to solving small-scale, synthetic problems, neglecting the challenges posed by real-world use cases. To bridge this gap, this work proposes a DRL-based Decision Support System (DSS) for real-world scale DTAPS. To this end, we introduce a DRL agent with two novel elements: a graph structure for observations and actions that can effectively represent any DTAP and a reward function that is provably equivalent to the objective of minimizing the average cycle time of tasks. The combination of these two novelties allows the agent to learn effective and generalizable assignment policies for real-world scale DTAPs. The proposed DSS is evaluated on five DTAP instances whose parameters are extracted from real-world logs through process mining. The experimental evaluation shows how the proposed DRL agent matches or outperforms the best baseline in all DTAP instances and generalizes on different time horizons and across instances.
Paper Structure (15 sections, 1 theorem, 9 equations, 10 figures, 5 tables)

This paper contains 15 sections, 1 theorem, 9 equations, 10 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Description
  4. Problem instance
  5. Problem State
  6. Problem Behavior
  7. A DSS for Solving DTAP at Scale
  8. DTAP as a Markov Decision Process
  9. Observations and Actions
  10. Actor and Critic Neural Networks
  11. Rewards
  12. Experimental Evaluation
  13. DTAP Parametrization
  14. Results
  15. Conclusions

Key Result

Theorem 1

Consider a DTAP with a limited time horizon $\tau_{\text{max}}$ and constant arrival rate of cases $\lambda$. Let $t_{\text{max}}$ be the sequence number of the last decision taken before $\tau_{\text{max}}$. Assuming the reward function proposed in eq:reward_func and a discount factor $\gamma = 1$,

Figures (10)

  • Figure 1: A BPMN representation of a loan application process.
  • Figure 2: A component model of the proposed DSS.
  • Figure 3: MDP interaction between agent and environment for DTAP.
  • Figure 4: A visual representation of the DTAP observation as a bipartite graph.
  • Figure 5: A visual representation of the DTAP observation as an assignment graph.
  • ...and 5 more figures

Theorems & Definitions (8)

  • Definition 3.1: Dynamic Task Assignment Problem
  • Definition 3.2: DTAP State
  • Definition 3.3: DTAP Transitions
  • Definition 4.1: Markov Decision Process
  • Definition 4.2: DTAP Observation
  • Definition 4.3: Reward Function that Minimizes Sum of Cycle Times
  • Theorem 1
  • proof