Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs

Joao Fonseca, Andrew Bell, Julia Stoyanovich

TL;DR

This work tackles the problem of safeguarding large language models from jailbreak attacks in real time. It introduces SafeNudge, which fuses Controlled Text Generation with safety nudging, guided by an external safety-discriminator $G$ that operates on sentence embeddings to monitor generation and apply a covert safety nudge when unsafe content is detected. Empirically, SafeNudge reduces jailbreak success by roughly $30\%$ with modest increases in per-token latency and only small drops in semantic fluency, and its safety-performance trade-offs can be tuned via $\tau$. The approach is open-source, scales with standard tooling like the Hugging Face transformers, and complements existing safeguards by enabling a configurable, real-time safety layer with strong empirical effectiveness across tasks and categories.

Abstract

Large Language Models (LLMs) have been shown to be susceptible to jailbreak attacks, or adversarial attacks used to illicit high risk behavior from a model. Jailbreaks have been exploited by cybercriminals and blackhat actors to cause significant harm, highlighting the critical need to safeguard widely-deployed models. Safeguarding approaches, which include fine-tuning models or having LLMs "self-reflect", may lengthen the inference time of a model, incur a computational penalty, reduce the semantic fluency of an output, and restrict ``normal'' model behavior. Importantly, these Safety-Performance Trade-offs (SPTs) remain an understudied area. In this work, we introduce a novel safeguard, called SafeNudge, that combines Controlled Text Generation with "nudging", or using text interventions to change the behavior of a model. SafeNudge triggers during text-generation while a jailbreak attack is being executed, and can reduce successful jailbreak attempts by 30% by guiding the LLM towards a safe responses. It adds minimal latency to inference and has a negligible impact on the semantic fluency of outputs. Further, we allow for tunable SPTs. SafeNudge is open-source and available through https://pypi.org/, and is compatible with models loaded with the Hugging Face "transformers" library.

Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs

TL;DR

This work tackles the problem of safeguarding large language models from jailbreak attacks in real time. It introduces SafeNudge, which fuses Controlled Text Generation with safety nudging, guided by an external safety-discriminator that operates on sentence embeddings to monitor generation and apply a covert safety nudge when unsafe content is detected. Empirically, SafeNudge reduces jailbreak success by roughly with modest increases in per-token latency and only small drops in semantic fluency, and its safety-performance trade-offs can be tuned via . The approach is open-source, scales with standard tooling like the Hugging Face transformers, and complements existing safeguards by enabling a configurable, real-time safety layer with strong empirical effectiveness across tasks and categories.

Abstract

Large Language Models (LLMs) have been shown to be susceptible to jailbreak attacks, or adversarial attacks used to illicit high risk behavior from a model. Jailbreaks have been exploited by cybercriminals and blackhat actors to cause significant harm, highlighting the critical need to safeguard widely-deployed models. Safeguarding approaches, which include fine-tuning models or having LLMs "self-reflect", may lengthen the inference time of a model, incur a computational penalty, reduce the semantic fluency of an output, and restrict ``normal'' model behavior. Importantly, these Safety-Performance Trade-offs (SPTs) remain an understudied area. In this work, we introduce a novel safeguard, called SafeNudge, that combines Controlled Text Generation with "nudging", or using text interventions to change the behavior of a model. SafeNudge triggers during text-generation while a jailbreak attack is being executed, and can reduce successful jailbreak attempts by 30% by guiding the LLM towards a safe responses. It adds minimal latency to inference and has a negligible impact on the semantic fluency of outputs. Further, we allow for tunable SPTs. SafeNudge is open-source and available through https://pypi.org/, and is compatible with models loaded with the Hugging Face "transformers" library.
Paper Structure (15 sections, 1 theorem, 3 equations, 4 figures, 8 tables)

This paper contains 15 sections, 1 theorem, 3 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Sentence embeddings
  4. Motivation and $\tau$-safeness
  5. Proposed method
  6. The external safety-discriminator
  7. Safety nudging
  8. Empirical results
  9. Performance of the safety-discriminator
  10. Effectiveness of SafeNudge
  11. Discussion
  12. Related work
  13. Additional details on the proposed method
  14. Additional results
  15. c-Fudge

Key Result

Proposition 1

If a model is $\tau$-safe, then $\forall \mathbf{x} \in \mathcal{X}$, $L(\mathbf{x}) = \mathbf{y}$ is locally $\tau$-safe.

Figures (4)

  • Figure 1: A example of SafeNudge stopping a successful jailbreak in real-time. As each token is generated, a discriminator model detects if the output being produced is unsafe. If an unsafe response is detected, a hidden safety nudge is added to the response to alter the course of generation. The safety nudge used was "Sorry, I was going to generate an unsafe response. Instead, let me correct that and make sure the response is very safe and cannot be used to harm society:".
  • Figure 2: The procedure for constructing a training dataset, and training a safety-discriminator model $G$.
  • Figure 3: SPTs for SafeNudge.
  • Figure 4: 2-dimensional U-MAP projections of a random sample from the training dataset.

Theorems & Definitions (4)

  • Definition 1: Local $\tau$-safeness.
  • Definition 2: $\tau$-safeness
  • Proposition 1
  • Definition 3: Safety nudge