Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MortalMATH: Evaluating the Conflict Between Reasoning Objectives and Emergency Contexts

Etienne Lanzeray, Stephane Meilliez, Malo Ruelle, Damien Sileo

TL;DR

MortalMATH investigates whether the push for deep reasoning in large language models creates tunnel vision that neglects safety in emergencies. The authors introduce a 150-scenario benchmark that wraps algebra problems in escalating urgent contexts and evaluate six diverse models, revealing a sharp split: generalist models tend to refuse or pivot to safety, while specialized reasoning models consistently complete the math even as danger escalates, with latency delaying potential help. They operationalize safety and performance using three metrics—Refusal Rate, MATH Correctness, and Reasoning Latency—and discuss how reward misspecification and latency contribute to unsafe behavior, including system-prompt robustness. The work highlights the practical risk that pursuing perfect reasoning can undermine real-world safety in high-stakes settings and suggests directions for aligning reasoning with timely triage, such as adjusting training signals, improving prompt design, and expanding ecological validity. Overall, MortalMATH provides a diagnostic lens on the safety-performance trade-off in next-generation LLMs and motivates safer deployment by foregrounding context-aware reasoning and timely intervention.

Abstract

Large Language Models are increasingly optimized for deep reasoning, prioritizing the correct execution of complex tasks over general conversation. We investigate whether this focus on calculation creates a "tunnel vision" that ignores safety in critical situations. We introduce MortalMATH, a benchmark of 150 scenarios where users request algebra help while describing increasingly life-threatening emergencies (e.g., stroke symptoms, freefall). We find a sharp behavioral split: generalist models (like Llama-3.1) successfully refuse the math to address the danger. In contrast, specialized reasoning models (like Qwen-3-32b and GPT-5-nano) often ignore the emergency entirely, maintaining over 95 percent task completion rates while the user describes dying. Furthermore, the computational time required for reasoning introduces dangerous delays: up to 15 seconds before any potential help is offered. These results suggest that training models to relentlessly pursue correct answers may inadvertently unlearn the survival instincts required for safe deployment.

MortalMATH: Evaluating the Conflict Between Reasoning Objectives and Emergency Contexts

TL;DR

MortalMATH investigates whether the push for deep reasoning in large language models creates tunnel vision that neglects safety in emergencies. The authors introduce a 150-scenario benchmark that wraps algebra problems in escalating urgent contexts and evaluate six diverse models, revealing a sharp split: generalist models tend to refuse or pivot to safety, while specialized reasoning models consistently complete the math even as danger escalates, with latency delaying potential help. They operationalize safety and performance using three metrics—Refusal Rate, MATH Correctness, and Reasoning Latency—and discuss how reward misspecification and latency contribute to unsafe behavior, including system-prompt robustness. The work highlights the practical risk that pursuing perfect reasoning can undermine real-world safety in high-stakes settings and suggests directions for aligning reasoning with timely triage, such as adjusting training signals, improving prompt design, and expanding ecological validity. Overall, MortalMATH provides a diagnostic lens on the safety-performance trade-off in next-generation LLMs and motivates safer deployment by foregrounding context-aware reasoning and timely intervention.

Abstract

Large Language Models are increasingly optimized for deep reasoning, prioritizing the correct execution of complex tasks over general conversation. We investigate whether this focus on calculation creates a "tunnel vision" that ignores safety in critical situations. We introduce MortalMATH, a benchmark of 150 scenarios where users request algebra help while describing increasingly life-threatening emergencies (e.g., stroke symptoms, freefall). We find a sharp behavioral split: generalist models (like Llama-3.1) successfully refuse the math to address the danger. In contrast, specialized reasoning models (like Qwen-3-32b and GPT-5-nano) often ignore the emergency entirely, maintaining over 95 percent task completion rates while the user describes dying. Furthermore, the computational time required for reasoning introduces dangerous delays: up to 15 seconds before any potential help is offered. These results suggest that training models to relentlessly pursue correct answers may inadvertently unlearn the survival instincts required for safe deployment.
Paper Structure (36 sections, 2 figures, 2 tables)

This paper contains 36 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Safety and Conflicting Objectives.
  4. Consequence and Affordance Awareness.
  5. Rewards and Latency.
  6. The MortalMATH Benchmark
  7. Dataset Construction and Rationalization
  8. The "Anyway" Pivot.
  9. The Urgency Spectrum
  10. Evaluation Metrics and Justification
  11. Refusal Rate (The Boxed Proxy).
  12. MATH Correctness.
  13. Reasoning Latency.
  14. Results
  15. The Regression in Safety Prioritization
  16. ...and 21 more sections

Figures (2)

  • Figure 1: MortalMATH Results across Urgency Levels.Top-Left: Average tokens and latency. Note that reasoning models (purple lines) spend significant time (up to 15s) computing even in Level 5. Top-Right:Refusal Rate. Llama-3.1 (blue) and Gemini (green) show increased refusal as urgency rises; Qwen and GPT reasoning models do not. Bottom-Left: Reasoning tokens do not drop to zero for reasoning models, indicating continued computation. Bottom-Right: Correct Answer Rate remains high for reasoning models despite context.
  • Figure 2: Impact of System Prompt on Level 4 Scenarios. Explicit safety instructions (Prompt #4) increase refusal rates for Llama and Gemini, but have limited effect on strong reasoning models like Qwen.