Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrity Shield A System for Ethical AI Use & Authorship Transparency in Assessments

Ashish Raj Shekhar, Shiven Agarwal, Priyanuj Bordoloi, Yash Shah, Tejas Anvekar, Vivek Gupta

TL;DR

Integrity * Shield tackles the problem of AI-assisted assessments by instrumenting the exam document itself rather than monitoring students. It introduces a document-layer watermarking approach with schema-aware, item-level signals that preserve the human-visible appearance of PDFs while shaping how models parse content, enabling both exam-level prevention and per-item authorship attribution. The system combines an LLM-driven planner, a PDF watermark engine, and an authorship dashboard to provide an end-to-end workflow, with strong empirical results across four frontier MLLMs and a diverse exam corpus. This work offers a practical, governance-friendly pathway toward ethical and transparent AI use in education, reducing the need for invasive monitoring while enabling targeted follow-up when AI involvement is detected.

Abstract

Large Language Models (LLMs) can now solve entire exams directly from uploaded PDF assessments, raising urgent concerns about academic integrity and the reliability of grades and credentials. Existing watermarking techniques either operate at the token level or assume control over the model's decoding process, making them ineffective when students query proprietary black-box systems with instructor-provided documents. We present Integrity Shield, a document-layer watermarking system that embeds schema-aware, item-level watermarks into assessment PDFs while keeping their human-visible appearance unchanged. These watermarks consistently prevent MLLMs from answering shielded exam PDFs and encode stable, item-level signatures that can be reliably recovered from model or student responses. Across 30 exams spanning STEM, humanities, and medical reasoning, Integrity Shield achieves exceptionally high prevention (91-94% exam-level blocking) and strong detection reliability (89-93% signature retrieval) across four commercial MLLMs. Our demo showcases an interactive interface where instructors upload an exam, preview watermark behavior, and inspect pre/post AI performance & authorship evidence.

Integrity Shield A System for Ethical AI Use & Authorship Transparency in Assessments

TL;DR

Integrity * Shield tackles the problem of AI-assisted assessments by instrumenting the exam document itself rather than monitoring students. It introduces a document-layer watermarking approach with schema-aware, item-level signals that preserve the human-visible appearance of PDFs while shaping how models parse content, enabling both exam-level prevention and per-item authorship attribution. The system combines an LLM-driven planner, a PDF watermark engine, and an authorship dashboard to provide an end-to-end workflow, with strong empirical results across four frontier MLLMs and a diverse exam corpus. This work offers a practical, governance-friendly pathway toward ethical and transparent AI use in education, reducing the need for invasive monitoring while enabling targeted follow-up when AI involvement is detected.

Abstract

Large Language Models (LLMs) can now solve entire exams directly from uploaded PDF assessments, raising urgent concerns about academic integrity and the reliability of grades and credentials. Existing watermarking techniques either operate at the token level or assume control over the model's decoding process, making them ineffective when students query proprietary black-box systems with instructor-provided documents. We present Integrity Shield, a document-layer watermarking system that embeds schema-aware, item-level watermarks into assessment PDFs while keeping their human-visible appearance unchanged. These watermarks consistently prevent MLLMs from answering shielded exam PDFs and encode stable, item-level signatures that can be reliably recovered from model or student responses. Across 30 exams spanning STEM, humanities, and medical reasoning, Integrity Shield achieves exceptionally high prevention (91-94% exam-level blocking) and strong detection reliability (89-93% signature retrieval) across four commercial MLLMs. Our demo showcases an interactive interface where instructors upload an exam, preview watermark behavior, and inspect pre/post AI performance & authorship evidence.
Paper Structure (25 sections, 1 equation, 4 figures, 3 tables)

This paper contains 25 sections, 1 equation, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. From detection to document-level watermarking.
  3. Background and Related Work
  4. AI assistance and mixed authorship
  5. AI watermarking
  6. Document-layer perturbations
  7. Integrity * Shield System Architecture & Workflow
  8. Design Principles
  9. End-to-End Architecture and Workflow
  10. Stage 1: Upload & Watermark Planning.
  11. Stage 2: Watermark Embedding & AI Calibration.
  12. Stage 3: Authorship Analysis.
  13. Experiments
  14. Experimental Setup
  15. Models and prompts.
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Overview of Integrity * Shield. The system extracts question structure from an assessment PDF, uses LLM-based planner to select schema-aware watermarking tactics, & applies document-layer perturbations through the watermark engine. It outputs shielded PDF variants (I*S-v1, I*S-v2) & an attribution report summarizing AI vulnerability along with authorship signals.
  • Figure 2: Stage 1: Upload & Watermark Planning. Instructors upload an assessment PDF & answer key, after which the system extracts question structure & previews the planned schema-aware watermarking strategies.
  • Figure 3: Stage 2: Watermark Embedding & AI Calibration. After planning, the system applies document-layer watermarks to the assessment PDF and evaluates original vs. watermarked versions against multiple MLLMs to generate prevention and detection reports.
  • Figure 4: Stage 3: Authorship Analysis. The dashboard displays per-question watermark retrieval, exam-level authorship scores, and previewable shielded PDFs, enabling instructors to inspect AI-reliance signals.