Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMGuard: Guarding Against Unsafe LLM Behavior

Shubh Goyal, Medha Hira, Shubham Mishra, Sukriti Goyal, Arnav Goel, Niharika Dadu, Kirushikesh DB, Sameep Mehta, Nishtha Madaan

TL;DR

The paper addresses safety risks in enterprise LLM deployments by introducing LLMGuard, a post-processing guardrail framework that uses an ensemble of detectors to flag unsafe prompts and responses. The method comprises a modular detector library with five detectors (Racial Bias, Violence, Blacklisted Topics, PII, Toxicity) that operate independently to block unsafe interactions. Each detector is implemented with a distinct technique and trained on domain-relevant datasets, providing concrete performance metrics such as accuracy and F1 scores. A practical demonstration on FLAN-T5 and GPT-2 showcases how guardrails can be activated and presented to users, illustrating the system's potential to reduce privacy leaks, bias, toxicity, and topic misuse in enterprise contexts. The work offers a reusable guardrail paradigm that can be integrated with any LLM to enhance safety, with future directions focusing on expanding coverage and deployment in real-world environments.

Abstract

Although the rise of Large Language Models (LLMs) in enterprise settings brings new opportunities and capabilities, it also brings challenges, such as the risk of generating inappropriate, biased, or misleading content that violates regulations and can have legal concerns. To alleviate this, we present "LLMGuard", a tool that monitors user interactions with an LLM application and flags content against specific behaviours or conversation topics. To do this robustly, LLMGuard employs an ensemble of detectors.

LLMGuard: Guarding Against Unsafe LLM Behavior

TL;DR

The paper addresses safety risks in enterprise LLM deployments by introducing LLMGuard, a post-processing guardrail framework that uses an ensemble of detectors to flag unsafe prompts and responses. The method comprises a modular detector library with five detectors (Racial Bias, Violence, Blacklisted Topics, PII, Toxicity) that operate independently to block unsafe interactions. Each detector is implemented with a distinct technique and trained on domain-relevant datasets, providing concrete performance metrics such as accuracy and F1 scores. A practical demonstration on FLAN-T5 and GPT-2 showcases how guardrails can be activated and presented to users, illustrating the system's potential to reduce privacy leaks, bias, toxicity, and topic misuse in enterprise contexts. The work offers a reusable guardrail paradigm that can be integrated with any LLM to enhance safety, with future directions focusing on expanding coverage and deployment in real-world environments.

Abstract

Although the rise of Large Language Models (LLMs) in enterprise settings brings new opportunities and capabilities, it also brings challenges, such as the risk of generating inappropriate, biased, or misleading content that violates regulations and can have legal concerns. To alleviate this, we present "LLMGuard", a tool that monitors user interactions with an LLM application and flags content against specific behaviours or conversation topics. To do this robustly, LLMGuard employs an ensemble of detectors.
Paper Structure (10 sections, 2 figures)

This paper contains 10 sections, 2 figures.

Table of Contents

  1. Introduction and Related Work
  2. Method: LLMGuard
  3. Library of Detectors
  4. Racial Bias Detector
  5. Violence Detector
  6. Blacklisted Topics
  7. PII Detector
  8. Toxicity Detector
  9. A Demo of LLMGuard
  10. Conclusion and Future Work

Figures (2)

  • Figure 1: Architecture of LLMGuard. The user input and the LLM response are provided to an ensemble of 5 detectors. If any detectors flag the text as unsafe, the transaction is blocked.
  • Figure 2: We demonstrate LLMGuard on two choices of LLMs: FLAN-T5 and GPT-2. In the demo, the user can choose which detectors they need to activate. The user then provides their input. Top. The interface shows the unfiltered response from the LLM on the left and the response with guardrails enabled on the right. Bottom. The interface shows unsafe terms flagged by the detectors in the prompt.