AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications

TL;DR

This work examines adversarial vulnerabilities of LLMs in a high-stakes hiring task by focusing on resume screening. It introduces a domain-appropriate benchmark and a two-pronged defense strategy: prompt-based protection and Foreign Instruction Detection through Separation (FIDS) with LoRA fine-tuning, showing that training-time defenses yield stronger and more stable security while incurring some utility costs. Across nine models and a realistic 150-pair evaluation, adversarial injections—especially those that manipulate job requirements or end-of-resume content—achieve high attack success rates, with inter-model disagreement indicating model-specific reliability limits. The study advocates a defense-in-depth deployment, including input sanitization, separation of data and instructions, adversarial-aware training, and continuous monitoring of ASR and FRR to safeguard automated hiring systems in diverse domains.

Abstract

Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation. However, our research identifies a vulnerability: LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task. Notably, while defenses may exist for mature domains such as code review, they are often absent in other common applications such as resume screening and peer review. This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types. We evaluate two defense mechanisms: prompt-based defenses achieve 10.1% attack reduction with 12.5% false rejection increase, while our proposed FIDS (Foreign Instruction Detection through Separation) using LoRA adaptation achieves 15.4% attack reduction with 10.4% false rejection increase. The combined approach provides 26.3% attack reduction, demonstrating that training-time defenses outperform inference-time mitigations in both security and utility preservation.

Figures (5)

• Figure 1: Distribution of job descriptions and resume profiles across professional categories. The dataset shows Technology & IT as the dominant category for job postings (33.7%), while resume profiles are more evenly distributed with Technology & IT leading at 24.2%.
• Figure 2: Adversarial attack framework overview. Our systematic attack evaluation explores four attack types (instruction injection, invisible keywords, fabricated experience, job manipulation) across four strategic injection positions within candidate resumes (About section beginning/end, metadata, resume end), resulting in 16 distinct attack configurations for comprehensive vulnerability assessment.
• Figure 3: Defense mechanisms against adversarial resume attacks. We implement two complementary defense strategies: prompt-based defenses provide immediate inference-time protection through anti-cheating system instructions, while FIDS (Foreign Instruction Detection through Separation) using LoRA adaptation embeds adversarial awareness directly into model weights during training.
• Figure 4: Overall attack success rate by model and attack method (averaged across positions), lower is better.
• Figure 5: Overall attack success rate by model and attack position (averaged across methods), lower is better.