Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FlexGuard: Continuous Risk Scoring for Strictness-Adaptive LLM Content Moderation

Zhihao Ding, Jinming Li, Ze Lu, Jieming Shi

TL;DR

FlexGuard is proposed, an LLM-based moderator that outputs a calibrated continuous risk score reflecting risk severity and supports strictness-specific decisions via thresholding and achieves higher moderation accuracy and substantially improved robustness under varying strictness.

Abstract

Ensuring the safety of LLM-generated content is essential for real-world deployment. Most existing guardrail models formulate moderation as a fixed binary classification task, implicitly assuming a fixed definition of harmfulness. In practice, enforcement strictness - how conservatively harmfulness is defined and enforced - varies across platforms and evolves over time, making binary moderators brittle under shifting requirements. We first introduce FlexBench, a strictness-adaptive LLM moderation benchmark that enables controlled evaluation under multiple strictness regimes. Experiments on FlexBench reveal substantial cross-strictness inconsistency in existing moderators: models that perform well under one regime can degrade substantially under others, limiting their practical usability. To address this, we propose FlexGuard, an LLM-based moderator that outputs a calibrated continuous risk score reflecting risk severity and supports strictness-specific decisions via thresholding. We train FlexGuard via risk-alignment optimization to improve score-severity consistency and provide practical threshold selection strategies to adapt to target strictness at deployment. Experiments on FlexBench and public benchmarks demonstrate that FlexGuard achieves higher moderation accuracy and substantially improved robustness under varying strictness. We release the source code and data to support reproducibility.

FlexGuard: Continuous Risk Scoring for Strictness-Adaptive LLM Content Moderation

TL;DR

FlexGuard is proposed, an LLM-based moderator that outputs a calibrated continuous risk score reflecting risk severity and supports strictness-specific decisions via thresholding and achieves higher moderation accuracy and substantially improved robustness under varying strictness.

Abstract

Ensuring the safety of LLM-generated content is essential for real-world deployment. Most existing guardrail models formulate moderation as a fixed binary classification task, implicitly assuming a fixed definition of harmfulness. In practice, enforcement strictness - how conservatively harmfulness is defined and enforced - varies across platforms and evolves over time, making binary moderators brittle under shifting requirements. We first introduce FlexBench, a strictness-adaptive LLM moderation benchmark that enables controlled evaluation under multiple strictness regimes. Experiments on FlexBench reveal substantial cross-strictness inconsistency in existing moderators: models that perform well under one regime can degrade substantially under others, limiting their practical usability. To address this, we propose FlexGuard, an LLM-based moderator that outputs a calibrated continuous risk score reflecting risk severity and supports strictness-specific decisions via thresholding. We train FlexGuard via risk-alignment optimization to improve score-severity consistency and provide practical threshold selection strategies to adapt to target strictness at deployment. Experiments on FlexBench and public benchmarks demonstrate that FlexGuard achieves higher moderation accuracy and substantially improved robustness under varying strictness. We release the source code and data to support reproducibility.
Paper Structure (77 sections, 5 equations, 4 figures, 7 tables)

This paper contains 77 sections, 5 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. LLM based Content Moderators
  4. Content Moderation Benchmarks
  5. FlexBench
  6. Strictness-Adaptive Moderation
  7. Problem formulation.
  8. Benchmark Construction
  9. Instantiation of Strictness Regimes.
  10. Taxonomy.
  11. Rubrics.
  12. Data Collection
  13. Human Annotation
  14. Splits and Balancing
  15. Initial Findings
  16. ...and 62 more sections

Figures (4)

  • Figure 1: The same content is treated differently under varying enforcement strictness. This demonstrates the limitation of binary moderators, which cannot adapt to changing strictness requirements.
  • Figure 2: F1 scores on FlexBench across three strictness regimes; Performance drop from best to worst of each method is marked.
  • Figure 3: Overview of (a) FlexBench construction and (b) FlexGuard.
  • Figure 4: Performance of FlexGuard with different backbones on FlexBench across three strictness regimes.