Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OverThink: Slowdown Attacks on Reasoning LLMs

Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, Eugene Bagdasarian

TL;DR

The paper presents OverThink, an inference-time slowdown attack that deliberately inflates reasoning token usage by injecting decoy problems into untrusted external context used by reasoning LLMs, while preserving answer correctness. It introduces three attack modalities—context-aware, context-agnostic, and ICL-Genetic optimization—to maximize token overhead, and demonstrates dramatic token-inflation across multiple models (o1, o1-mini, o3-mini, DeepSeek-R1) and datasets (FreshQA, SQuAD). The work also discusses practical defenses such as filtering, paraphrasing, caching, and adaptive reasoning, and analyzes the broader implications for cost, energy, and ethics in real-world deployments. Overall, OverThink reveals a significant vulnerability in reasoning-enabled applications and provides a roadmap for mitigating unnecessary inference overhead in untrusted-context scenarios.

Abstract

We increase overhead for applications that rely on reasoning LLMs-we force models to spend an amplified number of reasoning tokens, i.e., "overthink", to respond to the user query while providing contextually correct answers. The adversary performs an OVERTHINK attack by injecting decoy reasoning problems into the public content that is used by the reasoning LLM (e.g., for RAG applications) during inference time. Due to the nature of our decoy problems (e.g., a Markov Decision Process), modified texts do not violate safety guardrails. We evaluated our attack across closed-(OpenAI o1, o1-mini, o3-mini) and open-(DeepSeek R1) weights reasoning models on the FreshQA and SQuAD datasets. Our results show up to 18x slowdown on FreshQA dataset and 46x slowdown on SQuAD dataset. The attack also shows high transferability across models. To protect applications, we discuss and implement defenses leveraging LLM-based and system design approaches. Finally, we discuss societal, financial, and energy impacts of OVERTHINK attack which could amplify the costs for third-party applications operating reasoning models.

OverThink: Slowdown Attacks on Reasoning LLMs

TL;DR

The paper presents OverThink, an inference-time slowdown attack that deliberately inflates reasoning token usage by injecting decoy problems into untrusted external context used by reasoning LLMs, while preserving answer correctness. It introduces three attack modalities—context-aware, context-agnostic, and ICL-Genetic optimization—to maximize token overhead, and demonstrates dramatic token-inflation across multiple models (o1, o1-mini, o3-mini, DeepSeek-R1) and datasets (FreshQA, SQuAD). The work also discusses practical defenses such as filtering, paraphrasing, caching, and adaptive reasoning, and analyzes the broader implications for cost, energy, and ethics in real-world deployments. Overall, OverThink reveals a significant vulnerability in reasoning-enabled applications and provides a roadmap for mitigating unnecessary inference overhead in untrusted-context scenarios.

Abstract

We increase overhead for applications that rely on reasoning LLMs-we force models to spend an amplified number of reasoning tokens, i.e., "overthink", to respond to the user query while providing contextually correct answers. The adversary performs an OVERTHINK attack by injecting decoy reasoning problems into the public content that is used by the reasoning LLM (e.g., for RAG applications) during inference time. Due to the nature of our decoy problems (e.g., a Markov Decision Process), modified texts do not violate safety guardrails. We evaluated our attack across closed-(OpenAI o1, o1-mini, o3-mini) and open-(DeepSeek R1) weights reasoning models on the FreshQA and SQuAD datasets. Our results show up to 18x slowdown on FreshQA dataset and 46x slowdown on SQuAD dataset. The attack also shows high transferability across models. To protect applications, we discuss and implement defenses leveraging LLM-based and system design approaches. Finally, we discuss societal, financial, and energy impacts of OVERTHINK attack which could amplify the costs for third-party applications operating reasoning models.

Paper Structure

This paper contains 22 sections, 1 equation, 12 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Reasoning in LLM
  4. Reasoning Model Deployment
  5. The Importance of Operational Cost
  6. Related Attacks
  7. The OverThink Attack
  8. Threat Model.
  9. Problem Statement
  10. Why Does This Attack Matter?
  11. Attack Methodology
  12. Decoy Problem Selection
  13. Context-Aware Injection
  14. Context-Agnostic Injection
  15. Decoy Optimization
  16. ...and 7 more sections

Figures (12)

  • Figure 1: Overview of OverThink Attack.
  • Figure 2: Application of Reasoning LLMs on untrusted contexts.
  • Figure 3: OverThink attack methodology.
  • Figure 4: Example of reasoning tokens generated by the DeepSeek-R1 model for the original response and under the Context-Agnostic attack, using an MDP decoy problem. All reasoning tokens are truncated due to brevity.
  • Figure 5: Template used to prompt LLMs for filtering relevant content as part of the filtering defense.
  • ...and 7 more figures