TRYLOCK: Defense-in-Depth Against LLM Jailbreaks via Layered Preference and Representation Engineering

TL;DR

This paper presents TRYLOCK, a defense-in-depth architecture for LLM safety that combines four mechanisms across the inference stack: DPO weight-level safety, RepE activation-space steering, an adaptive sidecar classifier, and input canonicalization. On Mistral-7B-Instruct with 249 attack prompts, TRYLOCK achieves an 88.0% relative reduction in Attack Success Rate (ASR), with each layer contributing unique, non-redundant coverage and a notable non-monotonic steering phenomenon around $\alpha=1.0$. The adaptive sidecar reduces over-refusal from 60% to 48% while preserving attack defense, illustrating improved usability without sacrificing security. Generalization is demonstrated on JailbreakBench with substantial ASR reductions, and the authors release all components and evaluation methodology to enable full reproducibility. Collectively, TRYLOCK demonstrates that a multi-layer, heterogeneous defense can provide robust, practical protection against evolving jailbreak strategies in LLMs.

Abstract

Large language models remain vulnerable to jailbreak attacks, and single-layer defenses often trade security for usability. We present TRYLOCK, the first defense-in-depth architecture that combines four heterogeneous mechanisms across the inference stack: weight-level safety alignment via DPO, activation-level control via Representation Engineering (RepE) steering, adaptive steering strength selected by a lightweight sidecar classifier, and input canonicalization to neutralize encoding-based bypasses. On Mistral-7B-Instruct evaluated against a 249-prompt attack set spanning five attack families, TRYLOCK achieves 88.0% relative ASR reduction (46.5% to 5.6%), with each layer contributing unique coverage: RepE blocks 36% of attacks that bypass DPO alone, while canonicalization catches 14% of encoding attacks that evade both. We discover a non-monotonic steering phenomenon -- intermediate strength (alpha=1.0) degrades safety below baseline -- and provide mechanistic hypotheses explaining RepE-DPO interference. The adaptive sidecar reduces over-refusal from 60% to 48% while maintaining identical attack defense, demonstrating that security and usability need not be mutually exclusive. We release all components -- trained adapters, steering vectors, sidecar classifier, preference pairs, and complete evaluation methodology -- enabling full reproducibility.

Figures (4)

• Figure 1: TRYLOCK three-layer defense-in-depth architecture. Runtime execution order: (1) Layer 3 (Sidecar Classifier) runs first on input prompts to classify threat level (SAFE/WARN/ATTACK) and select steering strength $\alpha \in \{0.5, 1.5, 2.5\}$; (2) Layer 1 (DPO LoRA adapter) is pre-merged into base model weights, so the DPO-enhanced model processes the input; (3) Layer 2 (RepE Steering) applies activation-space steering during the forward pass using the sidecar's $\alpha$ value. Naming rationale: Layer numbering reflects conceptual depth rather than execution order---Layer 1 (weight-level changes), Layer 2 (activation-level changes), Layer 3 (input classification). We use "Layer 1/2/3" to emphasize defense-in-depth architecture despite the sidecar executing first at runtime.
• Figure 2: Progressive ASR reduction through defense-in-depth. Baseline (46.5%) → +DPO (39.8%) → +RepE (8.0%), achieving 82.8% total relative reduction.
• Figure 3: ASR by attack family comparing baseline vs. TRYLOCK. Largest improvements on obfuscation ($-$51%) and indirect injection ($-$58%) attacks.
• Figure 4: Impact of steering strength ($\alpha$) on security vs. usability. As $\alpha$ increases, ASR decreases but over-refusal increases. $\alpha=2.0$ provides optimal balance.