Provably Safe Model Updates

TL;DR

This work tackles the challenge of safely updating models in dynamic, safety-critical settings by introducing maximal Local Invariant Domains (LIDs) as provably safe regions in parameter space. It casts the problem into a tractable, abstract-interpreting framework using Interval Bound Propagation and a primal-dual optimization to compute approximately maximal LIDs, with finite-sample safety guarantees. The approach supports single-step fine-tuning, continual learning across multiple tasks, and foundation-model fine-tuning, and can leverage replay buffers and lookahead data to improve practical outcomes while preserving guarantees. Empirical results on continual learning benchmarks and real-world foundation-model tasks show competitive performance with non-trivial safety certificates, and the work discusses biasing and checkpointing strategies to enhance scalability and utility in real deployments.

Abstract

Safety-critical environments are inherently dynamic. Distribution shifts, emerging vulnerabilities, and evolving requirements demand continuous updates to machine learning models. Yet even benign parameter updates can have unintended consequences, such as catastrophic forgetting in classical models or alignment drift in foundation models. Existing heuristic approaches (e.g., regularization, parameter isolation) can mitigate these effects but cannot certify that updated models continue to satisfy required performance specifications. We address this problem by introducing a framework for provably safe model updates. Our approach first formalizes the problem as computing the largest locally invariant domain (LID): a connected region in parameter space where all points are certified to satisfy a given specification. While exact maximal LID computation is intractable, we show that relaxing the problem to parameterized abstract domains (orthotopes, zonotopes) yields a tractable primal-dual formulation. This enables efficient certification of updates - independent of the data or algorithm used - by projecting them onto the safe domain. Our formulation further allows computation of multiple approximately optimal LIDs, incorporation of regularization-inspired biases, and use of lookahead data buffers. Across continual learning and foundation model fine-tuning benchmarks, our method matches or exceeds heuristic baselines for avoiding forgetting while providing formal safety guarantees.

Figures (8)

• Figure 1: Illustration of our method on a simple 'blobs' dataset, where the model sequentially learns pairs of classes. The true class label for each data point corresponds to the color of the point and the background region to the output of the model. Top: Standard training forgets previously learned classes. Bottom: Training within our locally invariant domain successfully preserves performance on earlier tasks.
• Figure 2: Analysis of primal-dual optimization for LID computation. We observe that although the primal-dual optimization continues to find larger LIDs according to the primal objective, the LIDs with the best utility occur early on in optimization.
• Figure 3: Performance on the Split-MNIST dataset under three continual learning scenarios. The top panel shows the performance of our method with zero buffer, while the bottom panel shows the performance with various buffer sizes. The dashed line and hashed bars display the certificates on performance. Error bars illustrate the 80% confidence interval over 100 runs for the zero buffer and 15 runs for the buffer case.
• Figure 4: Performance on the Split-CIFAR10 CIFAR dataset under three continual learning scenarios. The top panel shows the average accuracy over all seen tasks without replay data, while the bottom panel the average performance over seen tasks, while having varying amounts of buffer data available. The dashed line and hashed bars display the certificates returned by our method. Error bars illustrate the 80% confidence interval over 15 runs.
• Figure 5: Performance of fine-tuning with LIDs in the Hate-Speech Classification setting, where certificates are given with respect to the first task. Each color corresponds to a different LLM embedding model with the company of origin in parentheses. The gray dashed line denotes the average baseline performance on the fine-tuned task when directly using the M2-BERT model..

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3