Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models

TL;DR

The paper introduces Antislop, a framework to detect and suppress repetitive, AI-specific 'slop' in language models via a three-pronged approach: an inference-time Antislop Sampler that backtracks to suppress unwanted patterns, an automated pipeline to profile model-specific slop against human baselines, and Final Token Preference Optimization (FTPO) that surgically implants suppression into model weights. Empirically, slop patterns can be extremely overrepresented and model-specific, yet FTPO achieves around 90% suppression with minimal or no loss in cross-domain tasks like GSM8K, MMLU, and long-form creative writing, outperforming token banning and Direct Preference Optimization (DPO). The sampler provides robust suppression (including regex-based bans) but incurs substantial throughput costs, motivating the automated data-generation + FTPO training pipeline. Overall, Antislop demonstrates that targeted, logit-space fine-tuning plus inference-time control can greatly reduce repetition while preserving model capabilities, with the code and results released under MIT license.

Abstract

Widespread LLM adoption has introduced characteristic repetitive phraseology, termed "slop," which degrades output quality and makes AI-generated text immediately recognizable. We present Antislop, a comprehensive framework providing tools to both detect and eliminate these overused patterns. Our approach combines three innovations: (1) The Antislop Sampler, which uses backtracking to suppress unwanted strings at inference time without destroying vocabulary; (2) An automated pipeline that profiles model-specific slop against human baselines and generates training data; (3) Final Token Preference Optimization (FTPO), a novel fine-tuning method that operates on individual tokens, surgically adjusting logits wherever a banned pattern has appeared in an inference trace. We demonstrate that some slop patterns appear over 1,000x more frequently in LLM output than human text. The Antislop Sampler successfully suppresses 8,000+ patterns while maintaining quality, whereas token banning becomes unusable at just 2,000. Most importantly, FTPO achieves 90% slop reduction while maintaining or improving performance in cross-domain evals including GSM8K, MMLU, and creative writing tasks. In contrast, DPO suffers significant degradation in writing quality and lexical diversity despite achieving weaker suppression. We release all code and results under MIT license: https://github.com/sam-paech/auto-antislop.

Figures (16)

• Figure 1: Pipeline for identifying and suppressing overused writing patterns in a language model.
• Figure 2: The Antislop backtracking mechanism detects unwanted patterns in the inference trace, backtracks to the first token of the banned sequence, lowers its probability, then resamples.
• Figure 3: FTPO achieves 90% slop suppression with minimal quality loss, outperforming DPO and token banning. The figure evaluates four suppression methods on gemma-3-12b across banlist sizes of 2k, 4k, and 8k patterns. FTPO maintains baseline writing quality while suppressing 85-90% of unwanted patterns. In contrast, DPO degrades quality by 6-15 points despite achieving only 80-82% suppression. Token banning shows catastrophic quality collapse. Error bars show 95% confidence intervals (CI95). $n=1{,}000$ outputs per condition.
• Figure 5: (a) Impact on writing quality from training to high preference accuracy targets; (b) Logit divergence from reference as training progresses.
• Figure 6: Our methods can suppress 90+ percent of banlist occurrences while allowing the banlist through when contextually necessary. Antislop Sampler, FTPO, DPO and token banning are compared on banlist suppression efficacy under normal writing conditions (non-adversarial prompts) and when the model is explicitly instructed to use the banned vocab (adversarial prompts). We indicate optimal behavior for most real-world use cases to be maximal suppression in normal writing conditions, and minimal (preferably zero) suppression in adversarial conditions -- i.e. when the model has no coherent alternatives.