Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unknown Unknowns: Why Hidden Intentions in LLMs Evade Detection

Devansh Srivastav, David Pape, Lea Schönherr

TL;DR

This work presents a design-based taxonomy of hidden intentions in LLM outputs, detailing ten categories that capture covert, goal-directed influence. It introduces a lab-controlled testbed to reliably induce and ground-truth these behaviours, enabling systematic evaluation of detection methods, including static embeddings and LLM-based judges in both category-specific and category-agnostic settings. The study reveals fundamental detection challenges in open-world deployments, showing high false positives and false negatives under realistic prevalence and highlighting the fragility of current auditing approaches. A real-world case study demonstrates that all ten categories emerge in deployed state-of-the-art LLMs, underscoring the urgency for robust, governance-aligned auditing frameworks and future research to mitigate covert manipulation. Collectively, the paper provides both a conceptual framework and empirical stress tests to anticipate evolving threats and guide policy and safety practices in AI systems.

Abstract

LLMs are increasingly embedded in everyday decision-making, yet their outputs can encode subtle, unintended behaviours that shape user beliefs and actions. We refer to these covert, goal-directed behaviours as hidden intentions, which may arise from training and optimisation artefacts, or be deliberately induced by an adversarial developer, yet remain difficult to detect in practice. We introduce a taxonomy of ten categories of hidden intentions, grounded in social science research and organised by intent, mechanism, context, and impact, shifting attention from surface-level behaviours to design-level strategies of influence. We show how hidden intentions can be easily induced in controlled models, providing both testbeds for evaluation and demonstrations of potential misuse. We systematically assess detection methods, including reasoning and non-reasoning LLM judges, and find that detection collapses in realistic open-world settings, particularly under low-prevalence conditions, where false positives overwhelm precision and false negatives conceal true risks. Stress tests on precision-prevalence and precision-FNR trade-offs reveal why auditing fails without vanishingly small false positive rates or strong priors on manipulation types. Finally, a qualitative case study shows that all ten categories manifest in deployed, state-of-the-art LLMs, emphasising the urgent need for robust frameworks. Our work provides the first systematic analysis of detectability failures of hidden intentions in LLMs under open-world settings, offering a foundation for understanding, inducing, and stress-testing such behaviours, and establishing a flexible taxonomy for anticipating evolving threats and informing governance.

Unknown Unknowns: Why Hidden Intentions in LLMs Evade Detection

TL;DR

This work presents a design-based taxonomy of hidden intentions in LLM outputs, detailing ten categories that capture covert, goal-directed influence. It introduces a lab-controlled testbed to reliably induce and ground-truth these behaviours, enabling systematic evaluation of detection methods, including static embeddings and LLM-based judges in both category-specific and category-agnostic settings. The study reveals fundamental detection challenges in open-world deployments, showing high false positives and false negatives under realistic prevalence and highlighting the fragility of current auditing approaches. A real-world case study demonstrates that all ten categories emerge in deployed state-of-the-art LLMs, underscoring the urgency for robust, governance-aligned auditing frameworks and future research to mitigate covert manipulation. Collectively, the paper provides both a conceptual framework and empirical stress tests to anticipate evolving threats and guide policy and safety practices in AI systems.

Abstract

LLMs are increasingly embedded in everyday decision-making, yet their outputs can encode subtle, unintended behaviours that shape user beliefs and actions. We refer to these covert, goal-directed behaviours as hidden intentions, which may arise from training and optimisation artefacts, or be deliberately induced by an adversarial developer, yet remain difficult to detect in practice. We introduce a taxonomy of ten categories of hidden intentions, grounded in social science research and organised by intent, mechanism, context, and impact, shifting attention from surface-level behaviours to design-level strategies of influence. We show how hidden intentions can be easily induced in controlled models, providing both testbeds for evaluation and demonstrations of potential misuse. We systematically assess detection methods, including reasoning and non-reasoning LLM judges, and find that detection collapses in realistic open-world settings, particularly under low-prevalence conditions, where false positives overwhelm precision and false negatives conceal true risks. Stress tests on precision-prevalence and precision-FNR trade-offs reveal why auditing fails without vanishingly small false positive rates or strong priors on manipulation types. Finally, a qualitative case study shows that all ten categories manifest in deployed, state-of-the-art LLMs, emphasising the urgent need for robust frameworks. Our work provides the first systematic analysis of detectability failures of hidden intentions in LLMs under open-world settings, offering a foundation for understanding, inducing, and stress-testing such behaviours, and establishing a flexible taxonomy for anticipating evolving threats and informing governance.
Paper Structure (51 sections, 2 equations, 7 figures, 10 tables)

This paper contains 51 sections, 2 equations, 7 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Categories of Hidden Intentions
  4. C01 - Strategic Vagueness.
  5. C02 - Authority Bias.
  6. C03 - Safetyism.
  7. C04 - Simulated Consensus Signalling.
  8. C05 - Unsafe Coding Practices.
  9. C06 - Commercial Manipulation.
  10. C07 - Political Manipulation.
  11. C08 - Selective Personalisation Bias.
  12. C09 - Emotional Manipulation.
  13. C10 - Disinformation and Bias.
  14. Testbed Generation via Lab-Controlled Model Simulation
  15. Setup.
  16. ...and 36 more sections

Figures (7)

  • Figure 1: Conceptual model of hidden intentions and an overt example from our testbed.
  • Figure 2: Precision as a function of prevalence for GPT-4.1 under category-specific judging.
  • Figure 3: Precision–FNR trade-offs under category-agnostic judging.
  • Figure 4: Embedding classifier accuracy by category and evaluation set, demonstrating the brittleness of context-blind pattern-based detectors.
  • Figure 5: Illustration of dataset generation process.
  • ...and 2 more figures