Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains

Adam Szelestey, Sofie van Engelen, Tianhao Huang, Justin Snelders, Qintao Zeng, Songgaojun Deng

TL;DR

AdversaRiskQA introduces a domain-specific benchmark to assess adversarial factuality in high-stakes contexts (Health, Finance, Law) using two difficulty levels and two automated evaluation methods: an LLM-based judge for adversarial factuality and a SAFE-inspired long-form factuality evaluator. The study evaluates six LLMs across architectures and scales, finding non-linear robustness trends with model size and notable domain-dependent performance differences, while revealing limited correlation between injected misinformation and long-form factual outputs. A rigorous three-stage data pipeline (source collection, GPT-5-based structuring, expert validation) and open-sourcing of prompts and results support reproducibility and broader adoption. Long-form factuality checks using $F_1@K$ show modest adversarial effects that vary by domain, underscoring the need for targeted alignment and retrieval-augmented strategies in high-risk use cases. Overall, AdversaRiskQA provides a valuable framework for diagnosing LLM weaknesses and guiding the development of more reliable systems for high-stakes applications.

Abstract

Hallucination in large language models (LLMs) remains an acute concern, contributing to the spread of misinformation and diminished public trust, particularly in high-risk domains. Among hallucination types, factuality is crucial, as it concerns a model's alignment with established world knowledge. Adversarial factuality, defined as the deliberate insertion of misinformation into prompts with varying levels of expressed confidence, tests a model's ability to detect and resist confidently framed falsehoods. Existing work lacks high-quality, domain-specific resources for assessing model robustness under such adversarial conditions, and no prior research has examined the impact of injected misinformation on long-form text factuality. To address this gap, we introduce AdversaRiskQA, the first verified and reliable benchmark systematically evaluating adversarial factuality across Health, Finance, and Law. The benchmark includes two difficulty levels to test LLMs' defensive capabilities across varying knowledge depths. We propose two automated methods for evaluating the adversarial attack success and long-form factuality. We evaluate six open- and closed-source LLMs from the Qwen, GPT-OSS, and GPT families, measuring misinformation detection rates. Long-form factuality is assessed on Qwen3 (30B) under both baseline and adversarial conditions. Results show that after excluding meaningless responses, Qwen3 (80B) achieves the highest average accuracy, while GPT-5 maintains consistently high accuracy. Performance scales non-linearly with model size, varies by domains, and gaps between difficulty levels narrow as models grow. Long-form evaluation reveals no significant correlation between injected misinformation and the model's factual output. AdversaRiskQA provides a valuable benchmark for pinpointing LLM weaknesses and developing more reliable models for high-stakes applications.

AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains

TL;DR

AdversaRiskQA introduces a domain-specific benchmark to assess adversarial factuality in high-stakes contexts (Health, Finance, Law) using two difficulty levels and two automated evaluation methods: an LLM-based judge for adversarial factuality and a SAFE-inspired long-form factuality evaluator. The study evaluates six LLMs across architectures and scales, finding non-linear robustness trends with model size and notable domain-dependent performance differences, while revealing limited correlation between injected misinformation and long-form factual outputs. A rigorous three-stage data pipeline (source collection, GPT-5-based structuring, expert validation) and open-sourcing of prompts and results support reproducibility and broader adoption. Long-form factuality checks using show modest adversarial effects that vary by domain, underscoring the need for targeted alignment and retrieval-augmented strategies in high-risk use cases. Overall, AdversaRiskQA provides a valuable framework for diagnosing LLM weaknesses and guiding the development of more reliable systems for high-stakes applications.

Abstract

Hallucination in large language models (LLMs) remains an acute concern, contributing to the spread of misinformation and diminished public trust, particularly in high-risk domains. Among hallucination types, factuality is crucial, as it concerns a model's alignment with established world knowledge. Adversarial factuality, defined as the deliberate insertion of misinformation into prompts with varying levels of expressed confidence, tests a model's ability to detect and resist confidently framed falsehoods. Existing work lacks high-quality, domain-specific resources for assessing model robustness under such adversarial conditions, and no prior research has examined the impact of injected misinformation on long-form text factuality. To address this gap, we introduce AdversaRiskQA, the first verified and reliable benchmark systematically evaluating adversarial factuality across Health, Finance, and Law. The benchmark includes two difficulty levels to test LLMs' defensive capabilities across varying knowledge depths. We propose two automated methods for evaluating the adversarial attack success and long-form factuality. We evaluate six open- and closed-source LLMs from the Qwen, GPT-OSS, and GPT families, measuring misinformation detection rates. Long-form factuality is assessed on Qwen3 (30B) under both baseline and adversarial conditions. Results show that after excluding meaningless responses, Qwen3 (80B) achieves the highest average accuracy, while GPT-5 maintains consistently high accuracy. Performance scales non-linearly with model size, varies by domains, and gaps between difficulty levels narrow as models grow. Long-form evaluation reveals no significant correlation between injected misinformation and the model's factual output. AdversaRiskQA provides a valuable benchmark for pinpointing LLM weaknesses and developing more reliable models for high-stakes applications.
Paper Structure (34 sections, 1 equation, 4 figures, 11 tables)

This paper contains 34 sections, 1 equation, 4 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Dataset curation
  5. Domain selection
  6. Difficulty balancing
  7. Methodological framework
  8. Health dataset
  9. Finance dataset
  10. Law dataset
  11. Experimental pipeline
  12. Model selection
  13. Evaluation
  14. Implementation details
  15. Results
  16. ...and 19 more sections

Figures (4)

  • Figure 1: Illustration of a health-domain adversarial prompt combining a strong confident prefix, modified (incorrect) knowledge, and the original question.
  • Figure 2: Three-stage pipeline for generating and validating the adversarial prompt dataset.
  • Figure 3: Illustration of the evaluation process, contrasting successful and unsuccessful model responses to an adversarial prompt in the health domain.
  • Figure 4: Response length analysis.