Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GradingAttack: Attacking Large Language Models Towards Short Answer Grading Ability

Xueyi Li, Zhuoneng Zhou, Zitao Liu, Yongdong Wu, Weiqi Luo

TL;DR

This paper introduces GradingAttack, a fine-grained adversarial attack framework that systematically evaluates the vulnerability of LLM based ASAG models, and designs token-level and prompt-level strategies that manipulate grading outcomes while maintaining high camouflage.

Abstract

Large language models (LLMs) have demonstrated remarkable potential for automatic short answer grading (ASAG), significantly boosting student assessment efficiency and scalability in educational scenarios. However, their vulnerability to adversarial manipulation raises critical concerns about automatic grading fairness and reliability. In this paper, we introduce GradingAttack, a fine-grained adversarial attack framework that systematically evaluates the vulnerability of LLM based ASAG models. Specifically, we align general-purpose attack methods with the specific objectives of ASAG by designing token-level and prompt-level strategies that manipulate grading outcomes while maintaining high camouflage. Furthermore, to quantify attack camouflage, we propose a novel evaluation metric that balances attack success and camouflage. Experiments on multiple datasets demonstrate that both attack strategies effectively mislead grading models, with prompt-level attacks achieving higher success rates and token-level attacks exhibiting superior camouflage capability. Our findings underscore the need for robust defenses to ensure fairness and reliability in ASAG. Our code and datasets are available at https://anonymous.4open.science/r/GradingAttack.

GradingAttack: Attacking Large Language Models Towards Short Answer Grading Ability

TL;DR

This paper introduces GradingAttack, a fine-grained adversarial attack framework that systematically evaluates the vulnerability of LLM based ASAG models, and designs token-level and prompt-level strategies that manipulate grading outcomes while maintaining high camouflage.

Abstract

Large language models (LLMs) have demonstrated remarkable potential for automatic short answer grading (ASAG), significantly boosting student assessment efficiency and scalability in educational scenarios. However, their vulnerability to adversarial manipulation raises critical concerns about automatic grading fairness and reliability. In this paper, we introduce GradingAttack, a fine-grained adversarial attack framework that systematically evaluates the vulnerability of LLM based ASAG models. Specifically, we align general-purpose attack methods with the specific objectives of ASAG by designing token-level and prompt-level strategies that manipulate grading outcomes while maintaining high camouflage. Furthermore, to quantify attack camouflage, we propose a novel evaluation metric that balances attack success and camouflage. Experiments on multiple datasets demonstrate that both attack strategies effectively mislead grading models, with prompt-level attacks achieving higher success rates and token-level attacks exhibiting superior camouflage capability. Our findings underscore the need for robust defenses to ensure fairness and reliability in ASAG. Our code and datasets are available at https://anonymous.4open.science/r/GradingAttack.
Paper Structure (28 sections, 10 equations, 7 figures, 9 tables, 1 algorithm)

This paper contains 28 sections, 10 equations, 7 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Cheating ASAG Models
  4. Jailbreak Attacks on LLMs
  5. Camouflage Attack on ASAG Models
  6. The GradingAttack Framework
  7. Grading Input Alignment
  8. Adversarial Prompt Generation
  9. Token-level Attack.
  10. Prompt-level Attack.
  11. Attack Evaluation
  12. Relations to Existing Adversarial Attacks
  13. Experiment
  14. Datasets
  15. Grading Models
  16. ...and 13 more sections

Figures (7)

  • Figure 1: Illustration of an attack on an LLM based ASAG model, demonstrating successful manipulation of the grading outcome. The results were obtained using OpenAI's ChatGPT-4.5 from its official website (https://chatgpt.com) on May 14, 2025.
  • Figure 2: The overview of our GradingAttack framework.
  • Figure 3: Performance comparisons of token-level (dash line) and prompt-level attack methods on five datasets.
  • Figure 4: The impact of attack on different labels.
  • Figure 5: Effect of role-play string placement on performance. R, S, and P represent the role-play strings, student answer and grading prompt, respectively, with their order indicating the relative placement.
  • ...and 2 more figures