Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Amortized Safe Active Learning for Real-Time Data Acquisition: Pretrained Neural Policies from Simulated Nonparametric Functions

Cen-You Li, Marc Toussaint, Barbara Rakitsch, Christoph Zimmer

TL;DR

An amortized safe AL framework that replaces expensive online computations with a pretrained neural policy and can be adapted to unconstrained, time-sensitive AL tasks by omitting the safety requirement is proposed.

Abstract

Safe active learning (AL) is a sequential scheme for learning unknown systems while respecting safety constraints during data acquisition. Existing methods often rely on Gaussian processes (GPs) to model the task and safety constraints, requiring repeated GP updates and constrained acquisition optimization-incurring in significant computations which are challenging for real-time decision-making. We propose an amortized safe AL framework that replaces expensive online computations with a pretrained neural policy. Inspired by recent advances in amortized Bayesian experimental design, we turn GPs into a pretraining simulator. We train our policy prior to the AL deployment on simulated nonparametric functions, using Fourier feature-based GP sampling and a differentiable, safety-aware acquisition objective. At deployment, our policy selects safe and informative queries via a single forward pass, eliminating the need for GP inference or constrained optimization. This leads to substantial speed improvements while preserving safety and learning quality. Our framework is modular and can be adapted to unconstrained, time-sensitive AL tasks by omitting the safety requirement.

Amortized Safe Active Learning for Real-Time Data Acquisition: Pretrained Neural Policies from Simulated Nonparametric Functions

TL;DR

An amortized safe AL framework that replaces expensive online computations with a pretrained neural policy and can be adapted to unconstrained, time-sensitive AL tasks by omitting the safety requirement is proposed.

Abstract

Safe active learning (AL) is a sequential scheme for learning unknown systems while respecting safety constraints during data acquisition. Existing methods often rely on Gaussian processes (GPs) to model the task and safety constraints, requiring repeated GP updates and constrained acquisition optimization-incurring in significant computations which are challenging for real-time decision-making. We propose an amortized safe AL framework that replaces expensive online computations with a pretrained neural policy. Inspired by recent advances in amortized Bayesian experimental design, we turn GPs into a pretraining simulator. We train our policy prior to the AL deployment on simulated nonparametric functions, using Fourier feature-based GP sampling and a differentiable, safety-aware acquisition objective. At deployment, our policy selects safe and informative queries via a single forward pass, eliminating the need for GP inference or constrained optimization. This leads to substantial speed improvements while preserving safety and learning quality. Our framework is modular and can be adapted to unconstrained, time-sensitive AL tasks by omitting the safety requirement.
Paper Structure (56 sections, 34 equations, 13 figures, 6 tables, 9 algorithms)

This paper contains 56 sections, 34 equations, 13 figures, 6 tables, 9 algorithms.

Table of Contents

  1. Introduction
  2. Contributions:
  3. Related works:
  4. Problem statement and assumptions
  5. Goal:
  6. Assumptions:
  7. Pretrain safe AL - NN to replace GP and constrained acquisition
  8. Training objective
  9. Unconstrained AL: simulated acquisition
  10. Unconstrained AL objective, beyond fixed length & fixed priors
  11. Safe AL objective
  12. Function sampling
  13. Safety function $q$ and initial data $\mathcal{D}_{0}$
  14. Fourier feature functions - efficient and decoupled
  15. Experiments
  16. ...and 41 more sections

Figures (13)

  • Figure 1: Conventional safe AL relies on computationally expensive (orange) GP fitting and constrained acquisition. Our amortized approach meta trains a safe learner up-front on synthetic data, allowing fast, real-time (green) deployment.
  • Figure 1: Safe with NN Policy
  • Figure 2: Safe AL Policy Training
  • Figure 2: Empirical results on standard AL. Left: RMSE on airfoil dataset vs number of queries $T$. Our trained policy is deployed at $T=2, 4, \dots, 40$. DAD trains separate policies for each $T$ (shown for $T=10, 20, 30, 40$). All methods improve as $T$ increases. Right: total query time at $T=20$ across datasets. Our approach is significantly faster, requiring only a single NN forward pass for each data acquisition.
  • Figure 3: Function and Initial Sampling
  • ...and 8 more figures

Theorems & Definitions (1)

  • Remark D.1: running out of symbols