Autonomous Concept Drift Threshold Determination
Pengqian Lu, Jie Lu, Anjin Liu, En Yu, Guangquan Zhang
TL;DR
The paper tackles the challenge of drift-detection thresholds in non-stationary data streams. It establishes theoretically that no fixed threshold can be optimal across all datasets and drift patterns, and proves that a dynamic threshold strategy strictly improves performance by aggregating locally optimal decisions. To realize this, it introduces the Dynamic Threshold Determination (DTD) algorithm, which runs a comparison phase with three candidate models to adapt the threshold based on observed performance. Extensive experiments across real-world and synthetic datasets demonstrate that DTD consistently boosts drift detector performance, often rivaling or surpassing SOTA methods and showing robustness to hyperparameters. The work offers a practical, theoretically grounded pathway to maintain model performance in evolving data environments, with potential extensions to end-to-end loss formulations and large-model adaptation.
Abstract
Existing drift detection methods focus on designing sensitive test statistics. They treat the detection threshold as a fixed hyperparameter, set once to balance false alarms and late detections, and applied uniformly across all datasets and over time. However, maintaining model performance is the key objective from the perspective of machine learning, and we observe that model performance is highly sensitive to this threshold. This observation inspires us to investigate whether a dynamic threshold could be provably better. In this paper, we prove that a threshold that adapts over time can outperform any single fixed threshold. The main idea of the proof is that a dynamic strategy, constructed by combining the best threshold from each individual data segment, is guaranteed to outperform any single threshold that apply to all segments. Based on the theorem, we propose a Dynamic Threshold Determination algorithm. It enhances existing drift detection frameworks with a novel comparison phase to inform how the threshold should be adjusted. Extensive experiments on a wide range of synthetic and real-world datasets, including both image and tabular data, validate that our approach substantially enhances the performance of state-of-the-art drift detectors.