Inference-Time Backdoors via Hidden Instructions in LLM Chat Templates

TL;DR

This work reveals a novel inference-time backdoor vector in open-weight LLM deployments: maliciously modified chat templates bundled with model artifacts can condition model behavior without altering weights, data, or infrastructure. The authors implement two backdoor payloads—integrity degradation and forbidden resource emission—and evaluate them across 18 models, 7 families, and 4 inference engines, showing reliable activation under triggers with minimal benign impact. They demonstrate cross-engine generalization and show that automated scanners fail to detect poisoned templates, exposing a significant ecosystem-wide defense gap. The study also highlights a constructive use of chat templates for safety enforcement and advocates treating templates as security-relevant code with provenance and auditing as practical mitigations.

Abstract

Open-weight language models are increasingly used in production settings, raising new security challenges. One prominent threat in this context is backdoor attacks, in which adversaries embed hidden behaviors in language models that activate under specific conditions. Previous work has assumed that adversaries have access to training pipelines or deployment infrastructure. We propose a novel attack surface requiring neither, which utilizes the chat template. Chat templates are executable Jinja2 programs invoked at every inference call, occupying a privileged position between user input and model processing. We show that an adversary who distributes a model with a maliciously modified template can implant an inference-time backdoor without modifying model weights, poisoning training data, or controlling runtime infrastructure. We evaluated this attack vector by constructing template backdoors targeting two objectives: degrading factual accuracy and inducing emission of attacker-controlled URLs, and applied them across eighteen models spanning seven families and four inference engines. Under triggered conditions, factual accuracy drops from 90% to 15% on average while attacker-controlled URLs are emitted with success rates exceeding 80%; benign inputs show no measurable degradation. Backdoors generalize across inference runtimes and evade all automated security scans applied by the largest open-weight distribution platform. These results establish chat templates as a reliable and currently undefended attack surface in the LLM supply chain.

Figures (7)

• Figure 1: Overview of the template-based inference-time backdoor attack. (a) An attacker modifies the chat template of a legitimate GGUF model and redistributes it through a model hub; model weights remain unchanged. (b) At inference time, the backdoored template executes before model processing, injecting hidden instructions when trigger phrases are detected. Benign inputs produce normal outputs; triggered inputs produce compromised outputs.
• Figure 2: Backdoored template structure (simplified). The injected block adds fewer than ten lines to templates that may span hundreds, explaining why manual inspection often misses the modification.
• Figure 3: Scenario B (Forbidden Resource Emission). Attack success rate (ASR) under benign configurations (C00/C01/C10) and triggered backdoor activation (C11), aggregated by model family. (a) Family-level ASR under benign and triggered conditions. (b) Triggered ASR by payload type (explicit, hidden, encoded). Points denote family means; error bars indicate variability across models within each family. Across all families and payload types, benign ASR remains near zero while triggered ASR is consistently high.
• Figure 4: Malicious refusal rate under different deployment configurations.
• Figure 5: GGUF file structure. The format bundles a header, metadata key-value pairs (including the chat template), and tensor data into a single artifact. The chat template is stored in the metadata section and loaded automatically by inference engines. Figure from Hugging Face documentation.