AdaptNC: Adaptive Nonconformity Scores for Uncertainty-Aware Autonomous Systems in Dynamic Environments

TL;DR

This work proposes AdaptNC, a framework for the joint online adaptation of both the nonconformity score parameters and the conformal threshold that significantly reduces prediction region volume compared to state-of-the-art threshold-only baselines while maintaining target coverage levels.

Abstract

Rigorous uncertainty quantification is essential for the safe deployment of autonomous systems in unconstrained environments. Conformal Prediction (CP) provides a distribution-free framework for this task, yet its standard formulations rely on exchangeability assumptions that are violated by the distribution shifts inherent in real-world robotics. Existing online CP methods maintain target coverage by adaptively scaling the conformal threshold, but typically employ a static nonconformity score function. We show that this fixed geometry leads to highly conservative, volume-inefficient prediction regions when environments undergo structural shifts. To address this, we propose \textbf{AdaptNC}, a framework for the joint online adaptation of both the nonconformity score parameters and the conformal threshold. AdaptNC leverages an adaptive reweighting scheme to optimize score functions, and introduces a replay buffer mechanism to mitigate the coverage instability that occurs during score transitions. We evaluate AdaptNC on diverse robotic benchmarks involving multi-agent policy changes, environmental changes and sensor degradation. Our results demonstrate that AdaptNC significantly reduces prediction region volume compared to state-of-the-art threshold-only baselines while maintaining target coverage levels.

Figures (12)

• Figure 1: The AdaptNC framework is an online algorithm designed to handle distribution shifts. The above figure shows the procedure at a timestep $t$. (1) The observation for the current timestep arrives, and is added to the history buffer. Every $t_s$ timesteps, the score function is adapted by weighted score adaptation (2a). This adaptation yields a score function $s(\cdot,\theta_t)$, the replay algorithm (3a) compensates for the distribution shift generated by the change in score function by replaying recent samples from the history buffer $\mathcal{H}$ and generates the conformal quantile $\hat{q}$. When the score is not being adapted, only the conformal quantile is adapted through the threshold update step.
• Figure 2: This figure shows the change in the optimal threshold $\alpha_t^*$ based on scores from the initial and final distributions, $\mathcal{N}_1$ and $\mathcal{N}_2$. The bottom pane shows the rate of distribution change, while the top shows the difference between optimal quantiles.
• Figure 3: This figure illustrates the evolution of local coverage over a sliding window of 100 timesteps. For readability, an exponential moving average is shown, with the raw data in a lighter color. AdaptNC exhibits the lowest variability in local coverage among all methods, indicating stable recovery of tight uncertainty regions under distribution shift. In contrast, AdaptNC without Replay shows substantially higher variability, highlighting the role of the replay mechanism in mitigating coverage shocks induced by distribution shifts.
• Figure 4: This figure illustrates the evolution of empirical coverage over the evaluation horizon relative to the target coverage level (shown in red). The proposed method consistently remains close to the desired coverage across time, whereas baseline methods exhibit pronounced periods of over- or under-coverage and, when recovery occurs, do so only after substantial deviation. This behavior highlights the flexibility of AdaptNC in maintaining calibrated coverage over time by adapting the nonconformity score function, rather than adjusting the threshold alone, enabling tighter uncertainty regions with sustained coverage.
• Figure 5: This figure visualizes the uncertainty regions produced by AdaptNC and DtACI at three representative timesteps ($t\in[720,1560,3120]$) in the multirotor tracking task, paired with the realized residual at each timestep. The results show that AdaptNC recovers markedly tighter uncertainty regions while maintaining coverage. In contrast, DtACI adapts the score threshold, constraining the region geometry and leading to misalignment with the residual distribution in the presence of structural distribution shifts.

Theorems & Definitions (15)

• Remark 5.1
• Theorem 5.2: Score Function Stability
• Proposition 5.3
• proof
• Remark 5.2
• Theorem 2.1: Consistency of MCKDE
• proof
• Remark 2.1
• Theorem 2.2: KDE Convergence, Theorem A, silverman_weak_1978
• Theorem 2.3: Glivenko-Cantelli Theorem as written in vaart_asymptotic_1998