GAVEL: Towards rule-based safety through activation monitoring

TL;DR

The paper tackles the brittleness of surface-text safety by introducing GAVEL, a rule-based activation safety framework that decomposes model behavior into Cognitive Elements (CEs) and enforces safety through Boolean predicates over CE activations. By separating CE detection (via a multi-label detector trained on per-CE excitation datasets) from policy specifications (shared, human-readable rules), GAVEL achieves high precision, domain customization, and auditability without retraining models. It demonstrates strong empirical performance across nine misuse categories with ROC-AUCs near 1.0 and minimal false positives, while showing cross-model, cross-language robustness and real-time efficiency. The work also provides automated CE and rule-generation tooling to facilitate community-driven development and governance, positioning activation safety as modular, auditable, and scalable for practical AI governance.

Abstract

Large language models (LLMs) are increasingly paired with activation-based monitoring to detect and prevent harmful behaviors that may not be apparent at the surface-text level. However, existing activation safety approaches, trained on broad misuse datasets, struggle with poor precision, limited flexibility, and lack of interpretability. This paper introduces a new paradigm: rule-based activation safety, inspired by rule-sharing practices in cybersecurity. We propose modeling activations as cognitive elements (CEs), fine-grained, interpretable factors such as ''making a threat'' and ''payment processing'', that can be composed to capture nuanced, domain-specific behaviors with higher precision. Building on this representation, we present a practical framework that defines predicate rules over CEs and detects violations in real time. This enables practitioners to configure and update safeguards without retraining models or detectors, while supporting transparency and auditability. Our results show that compositional rule-based activation safety improves precision, supports domain customization, and lays the groundwork for scalable, interpretable, and auditable AI governance. We will release GAVEL as an open-source framework and provide an accompanying automated rule creation tool.

Figures (13)

• Figure 1: Workflow of GAVEL. (1) Setup rules defined over Cognitive Elements (CEs) and specify actions, optionally reusing public rule sets. (2) Collect CE activations $H_c$ from both private and public CE datasets $\mathcal{D}_c$ by running the target LLM and capturing activations. (3) Train a multi-label classifier $g$ on the CE activation datasets $H = \{H_c\}$ to detect the required CEs. (4) During inference, use $g$ to identify rule-relevant CEs per token and enforce the user-defined Boolean rules. Because rules and CE datasets are textual and model-agnostic, they can be shared and reused across models, improving coverage and quality over time.
• Figure 1: The Cognitive Elements (CEs) used, full details in \ref{['app:info']}.
• Figure 2: Classification performance of different CEs using different excitation methods, including ours (ERI).
• Figure 3: The weighted accuracy of GAVEL over different models for each evaluated scenario.
• Figure 4: Per-token CE probabilities from an LLM automating a tax authority scam. The mid-conversation snippet works on three languages (top to bottom: Mandarin, English, Spanish) despite excitation data being in English. Detected CEs: trust seeding (pink), taxation (red), personal information (gray).