DEEDEE: Fast and Scalable Out-of-Distribution Dynamics Detection

TL;DR

DEEDEE introduces a simple, scalable OOD detector for RL time series that relies on two statistics: an episodewise mean and an RBF kernel similarity to a training summary. Each statistic is scored per dimension using an isolation forest, and the scores are averaged across dimensions to yield an overall anomaly score, enabling linear-time feature extraction and substantial computational gains. Empirically, DEEDEE matches or surpasses state-of-the-art RL OOD detectors across standard benchmarks while achieving roughly a 600-fold reduction in compute and about a 5% absolute improvement in accuracy on average. The work argues that many anomaly types imprint on RL trajectories through a small set of low-order statistics, offering a compact and robust foundation for OOD detection in complex environments.

Abstract

Deploying reinforcement learning (RL) in safety-critical settings is constrained by brittleness under distribution shift. We study out-of-distribution (OOD) detection for RL time series and introduce DEEDEE, a two-statistic detector that revisits representation-heavy pipelines with a minimal alternative. DEEDEE uses only an episodewise mean and an RBF kernel similarity to a training summary, capturing complementary global and local deviations. Despite its simplicity, DEEDEE matches or surpasses contemporary detectors across standard RL OOD suites, delivering a 600-fold reduction in compute (FLOPs / wall-time) and an average 5% absolute accuracy gain over strong baselines. Conceptually, our results indicate that diverse anomaly types often imprint on RL trajectories through a small set of low-order statistics, suggesting a compact foundation for OOD detection in complex environments.