Out-of-Distribution Detection for Continual Learning: Design Principles and Benchmarking

TL;DR

This work tackles the open problem of jointly handling continual learning and open-world out-of-distribution detection in dynamic environments. It introduces a unified taxonomy for CL and OOD methods and a benchmarking protocol that systematically evaluates many CL–OOD combinations using Avalanche and PyTorch-OOD. Through a large-scale study on CIFAR-100 with multiple task splits, it reveals robust patterns: inference-time OOD detectors are generally reliable across CL methods, while calibration-time and training-time detectors exhibit method-dependent trade-offs and potential synergy with certain CL strategies. The paper provides design principles and practical guidelines to build robust open-world continual learning systems and lays a foundation for future joint CL–OOD research.

Abstract

Recent years have witnessed significant progress in the development of machine learning models across a wide range of fields, fueled by increased computational resources, large-scale datasets, and the rise of deep learning architectures. From malware detection to enabling autonomous navigation, modern machine learning systems have demonstrated remarkable capabilities. However, as these models are deployed in ever-changing real-world scenarios, their ability to remain reliable and adaptive over time becomes increasingly important. For example, in the real world, new malware families are continuously developed, whereas autonomous driving cars are employed in many different cities and weather conditions. Models trained in fixed settings can not respond effectively to novel conditions encountered post-deployment. In fact, most machine learning models are still developed under the assumption that training and test data are independent and identically distributed (i.i.d.), i.e., sampled from the same underlying (unknown) distribution. While this assumption simplifies model development and evaluation, it does not hold in many real-world applications, where data changes over time and unexpected inputs frequently occur. Retraining models from scratch whenever new data appears is computationally expensive, time-consuming, and impractical in resource-constrained environments. These limitations underscore the need for Continual Learning (CL), which enables models to incrementally learn from evolving data streams without forgetting past knowledge, and Out-of-Distribution (OOD) detection, which allows systems to identify and respond to novel or anomalous inputs. Jointly addressing both challenges is critical to developing robust, efficient, and adaptive AI systems.

Figures (4)

• Figure 1: Conceptual Representation of a general post-hoc Out-of-Distribution method employed on a continuously trained model. In the absence of OOD detection, OOD samples: unk here, are misclassified into existing classes.
• Figure 2: AUROC of CL method combined with Post-hoc OOD detectors, measured on CIFAR100-10T. For each CL method and OOD detector, we indicate in brackets its category, as listed in \ref{['tab:supported_methods']}. Rows: the CL methods are ordered based on AIA, ACA, and AF. Columns: the post-hoc OOD detectors ordered based on the AUROC on the Cumulative method.
• Figure 3: Trade-off between AIA and AUROC for different CL methods on the CIFAR100-10T dataset, where the reported AUROC for each CL method corresponds to the best-performing OOD detector among those evaluated.
• Figure 4: AUROC of CL methods combined with training-time OOD integration and post-hoc OOD detectors, measured on CIFAR100-10T. Rows: CL methods combined with LogitNorm, Outlier Exposure (OE), or PixMix. Columns: post-hoc OOD detectors, split into inference-time (left) and calibration-time (right).