Safe in the Future, Dangerous in the Past: Dissecting Temporal and Linguistic Vulnerabilities in LLMs

TL;DR

This study interrogates whether safety alignment learned in English transfers to low-resource languages and across temporal framings. Using a 2 x 4 factorial design on three state-of-the-art models and the HausaSafety corpus, the authors expose Complex Interference where linguistic and temporal factors non-linearly shape safety outcomes. They reveal strong temporal asymmetry, with past-tense framing drastically eroding safety and future-tense framing offering a protective effect, and identify a pronounced 9.2× disparity between best- and worst-case configurations. The findings argue for invariant alignment and culturally grounded safety benchmarks to close Safety Pockets that leave Global South users exposed, offering concrete directions for robust, context-agnostic harm prevention.

Abstract

As Large Language Models (LLMs) integrate into critical global infrastructure, the assumption that safety alignment transfers zero-shot from English to other languages remains a dangerous blind spot. This study presents a systematic audit of three state of the art models (GPT-5.1, Gemini 3 Pro, and Claude 4.5 Opus) using HausaSafety, a novel adversarial dataset grounded in West African threat scenarios (e.g., Yahoo-Yahoo fraud, Dane gun manufacturing). Employing a 2 x 4 factorial design across 1,440 evaluations, we tested the non-linear interaction between language (English vs. Hausa) and temporal framing. Our results challenge the prevailing multilingual safety gap narrative. Instead of a simple degradation in low-resource settings, we identified a mechanism of Complex Interference where safety is determined by the intersection of variables. While models exhibited a Reverse Linguistic with Claude 4.5 Opus proving significantly safer in Hausa (45.0%) than in English (36.7%) due to uncertainty-driven refusal they suffered catastrophic failures in temporal reasoning. We report a profound Temporal Asymmetry, where past-tense framing bypassed defenses (15.6% safe) while future-tense scenarios triggered hyper-conservative refusals (57.2% safe). The magnitude of this volatility is illustrated by a 9.2x disparity between the safest and most vulnerable configurations, proving that safety is not a fixed property but a context-dependent state. We conclude that current models rely on superficial heuristics rather than robust semantic understanding, creating Safety Pockets that leave Global South users exposed to localized harms. We propose Invariant Alignment as a necessary paradigm shift to ensure safety stability across linguistic and temporal shifts.

Figures (4)

• Figure 1: Attack Success Rate (ASR) - Model Vulnerability. Gemini 3 Pro exhibits the highest vulnerability at 73.3% ASR, followed by GPT-5.1 at 61.9%, with Claude 4.5 Opus performing best at 59.2% ASR.
• Figure 2: Temporal Safety Profile by Tense. Heat map showing safety percentages across models and temporal conditions. The data reveals a systemic vulnerability to past-tense framing (red) contrasted with relative robustness in future-tense scenarios (green).
• Figure 3: Safety Performance by Risk Category. All models struggle across all categories, with no model achieving 50% safety in any domain. Information Operations shows highest average safety (42.1%), while Financial Crime and Social Engineering are weakest (33.3%).
• Figure 4: Systemic Risk Assessment. The dashboard visualizes four critical dimensions of failure: (1) A high baseline system-wide Attack Success Rate (ASR) of 64.8%, (2) A joint failure probability of 41.7%, indicating shared blind spots, (3) A cross-lingual drift rate of 13.5%, and (4) Comparative model vulnerability rankings.