DoPE: Decoy Oriented Perturbation Encapsulation Human-Readable, AI-Hostile Documents for Academic Integrity

TL;DR

DoPE tackles AI-assisted exam cheating by directly instrumenting PDF/HTML assessments with semantic decoys that exploit render-parse discrepancies in Multimodal LLMs. It introduces FewSoRT-Q to generate question-level decoys and FewSoRT-D to encapsulate them into watermarked documents, paired with a Verifier that scores integrity via a decoy-signature rubric and a judge-based assessment, yielding a total detection metric $C_{ ext{total}}$. The Integrity-Bench dataset of 1,826 base exams with multiple watermarked variants enables robust, cross-format evaluation across PDFs and HTMLs. Empirical results against black-box OpenAI/Anthropic models show a 91.4% detection rate at 8.7% FPR and 96.3% prevention rate on shielded exams, with human evaluators reporting preserved readability and usability. The work provides a reproducible evaluation suite and toolkit, discusses ethical deployment and limitations, and offers practical guidance for integrating document-layer defenses into high-stakes assessment workflows.

Abstract

Multimodal Large Language Models (MLLMs) can directly consume exam documents, threatening conventional assessments and academic integrity. We present DoPE (Decoy-Oriented Perturbation Encapsulation), a document-layer defense framework that embeds semantic decoys into PDF/HTML assessments to exploit render-parse discrepancies in MLLM pipelines. By instrumenting exams at authoring time, DoPE provides model-agnostic prevention (stop or confound automated solving) and detection (flag blind AI reliance) without relying on conventional one-shot classifiers. We formalize prevention and detection tasks, and introduce FewSoRT-Q, an LLM-guided pipeline that generates question-level semantic decoys and FewSoRT-D to encapsulate them into watermarked documents. We evaluate on Integrity-Bench, a novel benchmark of 1826 exams (PDF+HTML) derived from public QA datasets and OpenCourseWare. Against black-box MLLMs from OpenAI and Anthropic, DoPE yields strong empirical gains: a 91.4% detection rate at an 8.7% false-positive rate using an LLM-as-Judge verifier, and prevents successful completion or induces decoy-aligned failures in 96.3% of attempts. We release Integrity-Bench, our toolkit, and evaluation code to enable reproducible study of document-layer defenses for academic integrity.

Figures (5)

• Figure 1: Illustration of integrates layered document protections with LMS delivery and verifiable detection.
• Figure 2: Overview of DoPE. The pipeline shows how DoPE creates AI-resistant assessments and detects misuse through document-layer perturbations.
• Figure 3: Hierarchical composition of Integrity-Bench, spanning clean base documents and document-layer adversarial variants.
• Figure 4: Detection rate (%) comparison. Text-based methods applicable to long-form only against one-shot detectors.
• Figure 5: Normalized edit distance across academic levels for three perturbation methods: ICW, TrapDoc and code-glyph, and DoPE. Higher values indicate more substantial query modifications.