Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Operating Imperfect AI: Reliability Drift and Human Congestion

Ziyao Wang, Svetlozar T Rachev

TL;DR

This work addresses the operational gap in deploying imperfect AI by modeling HITL systems as a continuous-time queueing control problem with reliability drift. It develops a CTMDP formulation where the controller chooses a state-dependent risk threshold $T^*(q,\theta)$ to balance automated risk and human congestion, and proves the optimal policy is a threshold rule with the shadow price of capacity guiding decisions. Two core structural results—Congestion Shedding (threshold rises with backlog) and Safety Buffering (threshold falls during drift under a dominance condition)—yield intuitive managerial rules and a Capacity Phase Transition that delineates feasible versus unsafe regimes. Through simulations, the Optimal Dynamic Policy (ODP) outperforms static and drift-blind baselines, reducing costs and improving safety while demonstrating the practical value of dynamic reliability management. The findings offer concrete guidelines for operators: adapt thresholds to backlog, act as a safety buffer during drift, and recognize capacity boundaries that require provisioning or demand throttling when violated.

Abstract

The deployment of machine learning in high-stakes services relies on ``human-in-the-loop'' architectures to mitigate algorithmic uncertainty. However, existing static policies fail to address a fundamental tension: algorithms suffer from stochastic ``reliability drift,'' while human override capacity is scarce and congestible. We formulate the management of such systems as a dynamic queueing control problem. The system state is defined by the tuple (queue backlog, reliability regime), and the control variable is a state-dependent risk threshold. We prove that the optimal escalation policy is driven by the endogenous ``Shadow Price of Capacity.'' We establish two key structural monotonicity results: (i) Congestion Shedding, where the threshold rises with backlog to sacrifice marginal accuracy for responsiveness; and (ii) Safety Buffering, where the threshold lowers during drift to use the queue as a ``risk capacitor.'' Furthermore, we identify a critical ``Capacity Phase Transition'' in the arrival-drift parameter space, beyond which no policy can maintain safety standards without causing structural system failure (infinite queues). Our results provide rigorous operational rules for managing the interface between imperfect algorithms and congested experts.

Operating Imperfect AI: Reliability Drift and Human Congestion

TL;DR

This work addresses the operational gap in deploying imperfect AI by modeling HITL systems as a continuous-time queueing control problem with reliability drift. It develops a CTMDP formulation where the controller chooses a state-dependent risk threshold to balance automated risk and human congestion, and proves the optimal policy is a threshold rule with the shadow price of capacity guiding decisions. Two core structural results—Congestion Shedding (threshold rises with backlog) and Safety Buffering (threshold falls during drift under a dominance condition)—yield intuitive managerial rules and a Capacity Phase Transition that delineates feasible versus unsafe regimes. Through simulations, the Optimal Dynamic Policy (ODP) outperforms static and drift-blind baselines, reducing costs and improving safety while demonstrating the practical value of dynamic reliability management. The findings offer concrete guidelines for operators: adapt thresholds to backlog, act as a safety buffer during drift, and recognize capacity boundaries that require provisioning or demand throttling when violated.

Abstract

The deployment of machine learning in high-stakes services relies on ``human-in-the-loop'' architectures to mitigate algorithmic uncertainty. However, existing static policies fail to address a fundamental tension: algorithms suffer from stochastic ``reliability drift,'' while human override capacity is scarce and congestible. We formulate the management of such systems as a dynamic queueing control problem. The system state is defined by the tuple (queue backlog, reliability regime), and the control variable is a state-dependent risk threshold. We prove that the optimal escalation policy is driven by the endogenous ``Shadow Price of Capacity.'' We establish two key structural monotonicity results: (i) Congestion Shedding, where the threshold rises with backlog to sacrifice marginal accuracy for responsiveness; and (ii) Safety Buffering, where the threshold lowers during drift to use the queue as a ``risk capacitor.'' Furthermore, we identify a critical ``Capacity Phase Transition'' in the arrival-drift parameter space, beyond which no policy can maintain safety standards without causing structural system failure (infinite queues). Our results provide rigorous operational rules for managing the interface between imperfect algorithms and congested experts.
Paper Structure (38 sections, 5 theorems, 25 equations, 3 figures, 1 table)

This paper contains 38 sections, 5 theorems, 25 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. The Operational Reality of AI Deployment
  3. The Economic Gap: From Fixed Costs to Shadow Prices
  4. The Core Tension: Congestion and Reliability Drift
  5. Research Questions and Methodology
  6. Literature Review
  7. Human-AI Collaboration and Hybrid Intelligence
  8. Admission Control and Triage in Service Systems
  9. Operational Adaptation to Model Drift
  10. Positioning and Summary
  11. Model Formulation
  12. Task Arrivals and Heterogeneous Risk Profiles
  13. Reliability Drift Dynamics
  14. The Cost of Automation
  15. The Human Service Queue
  16. ...and 23 more sections

Key Result

Theorem 1

For any system state $(q, \theta) \in \mathbb{N}_0 \times \Theta$, there exists a unique risk threshold $T^*(q, \theta) \in [0, 1] \cup \{\infty\}$ such that the optimal policy is given by: Specifically, the threshold is defined by the inverse cost function: where $c_{\text{auto}}^{-1}(y \mid \theta) = \inf \{ s \in \mathcal{S} : c_{\text{auto}}(s, \theta) \ge y \}$. If the set is empty, $T^*(q,

Figures (3)

  • Figure 1: Structure of the Optimal Policy: The "Drift Switch". The blue curve represents the stable state (State L), while the red dashed curve represents the drift state (State H), visually demonstrating the Safety Buffering effect.
  • Figure 2: System Stability & Capacity Phase Transition. The heatmap shows the capacity headroom. The black line marks the structural boundary between the stable region (manageable) and the unstable region (structural deficit).
  • Figure 3: The Value of Agility (Managerial Insight). As the environment becomes more turbulent (higher drift intensity), the relative cost savings of the Optimal Policy versus the Static Policy increase, justifying the investment in ML Ops.

Theorems & Definitions (16)

  • Remark 1: Observability of Risk Scores
  • Remark 2: Discounted vs. Average Cost
  • Theorem 1: Optimality of Threshold Policies
  • proof
  • Remark 3: Operational Interpretation
  • Proposition 1: Convexity of Value Function
  • proof
  • Theorem 2: Congestion Shedding
  • proof
  • Remark 4: Economic Implications of Congestion Shedding
  • ...and 6 more