Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Drift-to-Action Controllers: Budgeted Interventions with Online Risk Certificates

Ismail Lamaakal, Chaymae Yahyati, Khalid El Makkaoui, Ibrahim Ouahbi, Yassine Maleh

TL;DR

Overall, online risk certification enables reliable drift response and reframes monitoring as decision-making with safety and reframes monitoring as decision-making with safety.

Abstract

Deployed machine learning systems face distribution drift, yet most monitoring pipelines stop at alarms and leave the response underspecified under labeling, compute, and latency constraints. We introduce Drift2Act, a drift-to-action controller that treats monitoring as constrained decision-making with explicit safety. Drift2Act combines a sensing layer that maps unlabeled monitoring signals to a belief over drift types with an active risk certificate that queries a small set of delayed labels from a recent window to produce an anytime-valid upper bound $U_t(δ)$ on current risk. The certificate gates operation: if $U_t(δ) \le τ$, the controller selects low-cost actions (e.g., recalibration or test-time adaptation); if $U_t(δ) > τ$, it activates abstain/handoff and escalates to rollback or retraining under cooldowns. In a realistic streaming protocol with label delay and explicit intervention costs, Drift2Act achieves near-zero safety violations and fast recovery at moderate cost on WILDS Camelyon17, DomainNet, and a controlled synthetic drift stream, outperforming alarm-only monitoring, adapt-always adaptation, schedule-based retraining, selective prediction alone, and an ablation without certification. Overall, online risk certification enables reliable drift response and reframes monitoring as decision-making with safety.

Drift-to-Action Controllers: Budgeted Interventions with Online Risk Certificates

TL;DR

Overall, online risk certification enables reliable drift response and reframes monitoring as decision-making with safety and reframes monitoring as decision-making with safety.

Abstract

Deployed machine learning systems face distribution drift, yet most monitoring pipelines stop at alarms and leave the response underspecified under labeling, compute, and latency constraints. We introduce Drift2Act, a drift-to-action controller that treats monitoring as constrained decision-making with explicit safety. Drift2Act combines a sensing layer that maps unlabeled monitoring signals to a belief over drift types with an active risk certificate that queries a small set of delayed labels from a recent window to produce an anytime-valid upper bound on current risk. The certificate gates operation: if , the controller selects low-cost actions (e.g., recalibration or test-time adaptation); if , it activates abstain/handoff and escalates to rollback or retraining under cooldowns. In a realistic streaming protocol with label delay and explicit intervention costs, Drift2Act achieves near-zero safety violations and fast recovery at moderate cost on WILDS Camelyon17, DomainNet, and a controlled synthetic drift stream, outperforming alarm-only monitoring, adapt-always adaptation, schedule-based retraining, selective prediction alone, and an ablation without certification. Overall, online risk certification enables reliable drift response and reframes monitoring as decision-making with safety.
Paper Structure (93 sections, 2 theorems, 156 equations, 2 figures, 14 tables, 1 algorithm)

This paper contains 93 sections, 2 theorems, 156 equations, 2 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sensing Layer: Monitoring Signals and Belief over Drift Type
  4. Monitoring windows and representations
  5. Evidence signals
  6. Belief over drift type
  7. Active Risk Certificate: Online Safety Layer
  8. Windowed risk under delayed supervision
  9. Active label sampling within the window
  10. Anytime-valid upper confidence bound
  11. Safety gating and escalation
  12. Label efficiency through targeted verification
  13. Safety guarantee
  14. Drift-to-Action Controller
  15. Decision variables and objective
  16. ...and 78 more sections

Key Result

Lemma A.1

(Anytime-valid upper confidence sequence via stitching). For any $\delta\in(0,1)$ and the radius $\mathrm{rad}(n,\delta)$ defined above, it holds that In this probability statement, the event inside holds simultaneously for all sample sizes $n$, which implies validity under optional stopping.

Figures (2)

  • Figure 1: Belief evolution over drift types. Posterior $b_t(d)=\mathbb{P}(D_t=d\mid z_{1:t})$ over $\{\mathrm{none},\mathrm{covariate},\mathrm{concept},\mathrm{subgroup}\}$ across time; dashed lines mark drift events that shift belief and guide intervention choice.
  • Figure 2: Certified controller performance under cost--safety trade-offs and drift. (Left) Safety--cost Pareto frontier on Camelyon17. (Right) Recovery dynamics around a drift event relative to threshold $\tau$, with intervention times marked.

Theorems & Definitions (2)

  • Lemma A.1
  • Theorem A.2