Defensive M2S: Training Guardrail Models on Compressed Multi-turn Conversations

TL;DR

Defensive M2S tackles the high cost of safeguarding long multi-turn conversations by training guardrails on M2S-compressed inputs instead of full histories. The approach compresses conversations into single-turn representations using hyphenize, numberize, or pythonize templates, retaining only user turns, which reduces training and inference costs from $O(n^2)$ to $O(n)$ while maintaining or improving recall for certain model-template pairs. Empirical evaluation across three guardrail families (LlamaGuard, Nemotron, Qwen3Guard) and SafeDialBench demonstrates substantial token reductions (up to $94\%$) and significant recall gains for favorable combinations (e.g., Qwen3Guard with hyphenize achieving $93.8\%$ recall). The results suggest single-template training is generally superior to mixed-template training, with important caveats about model compatibility, dataset scope, and potential context loss; future work should explore adaptive templates, larger training data, and broader benchmarks to validate real-world deployment viability.

Abstract

Guardrail models are essential for ensuring the safety of Large Language Model (LLM) deployments, but processing full multi-turn conversation histories incurs significant computational cost. We propose Defensive M2S, a training paradigm that fine-tunes guardrail models on Multi-turn to Single-turn (M2S) compressed conversations rather than complete dialogue histories. We provide a formal complexity analysis showing that M2S reduces training cost from $O(n^2)$ to $O(n)$ for $n$-turn conversations. Empirically, on our training dataset (779 samples, avg. 10.6 turns), M2S requires only 169K tokens compared to 15.7M tokens for the multi-turn baseline -- a 93$\times$ reduction. We evaluate Defensive M2S across three guardrail model families (LlamaGuard, Nemotron, Qwen3Guard) and three compression templates (hyphenize, numberize, pythonize) on SafeDialBench, a comprehensive multi-turn jailbreak benchmark. Our best configuration, Qwen3Guard with hyphenize compression, achieves 93.8% attack detection recall while reducing inference tokens by 94.6% (from 3,231 to 173 tokens per conversation). This represents a 38.9 percentage point improvement over the baseline while dramatically reducing both training and inference costs. Our findings demonstrate that M2S compression can serve as an effective efficiency technique for guardrail deployment, enabling scalable safety screening of long multi-turn conversations.