Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detecting LLM-Assisted Academic Dishonesty using Keystroke Dynamics

Atharva Mehta, Rajesh Kumar, Aman Singla, Kartik Bisht, Yaman Kumar Singla, Rajiv Ratn Shah

TL;DR

This work demonstrates that keystroke dynamics, as a behavioral signal, can robustly differentiate human-authored from AI-assisted writing when used alongside text features. By expanding datasets and introducing paraphrasing, the authors benchmark gradient-boosted detectors (LightGBM, CatBoost) and a Siamese-TypeNet model, revealing that ML models excel in structured contexts while TypeNet excels in paraphrase scenarios. A deception threat model shows forged keystrokes can degrade performance, which the authors counter with adversarial training, achieving strong robustness. Across comparisons with DetectGPT, LLaMA-3.3-70B-Instruct, and human evaluators, purely text-based or subjective judgments perform near chance, highlighting the value of incorporating process-level data for academic integrity assessment. The study suggests a practical, multimodal approach to deter GenAI-assisted cheating while acknowledging limitations and ethical considerations, with future work focusing on larger, diverse datasets and additional behavioral cues.

Abstract

The rapid adoption of generative AI tools has intensified the challenge of maintaining academic integrity. Conventional plagiarism detectors, which rely on text-matching or text-intrinsic features, often fail to identify submissions that have been AI-assisted or paraphrased. To address this limitation, we introduce keystroke-dynamics-based detectors that analyze how, rather than what, a person writes to distinguish genuine from assisted writing. Building on our earlier study, which collected keystroke data from 40 participants and trained a modified TypeNet model to detect assisted text, we expanded the dataset by adding 90 new participants and introducing a paraphrasing-based plagiarism-detection mode. We then benchmarked two additional gradient-boosting classifiers, LightGBM and CatBoost, alongside TypeNet, and compared their performance with DetectGPT, LLaMA 3.3 70B Instruct, and the results of 44 human evaluators. To further assess and improve robustness, we proposed a deception-based threat model simulating forged keystrokes and applied adversarial training as a countermeasure. Results show that the machine learning models achieve F1 scores above 97% in structured settings, while TypeNet performs best in detecting paraphrasing, with an F1 score of 86.9%. In contrast, text-only detectors and human evaluators perform near-chance, demonstrating that keystroke dynamics provide a strong behavioral signal for identifying AI-assisted plagiarism and support the use of multimodal behavioral features for reliable academic integrity assessment.

Detecting LLM-Assisted Academic Dishonesty using Keystroke Dynamics

TL;DR

This work demonstrates that keystroke dynamics, as a behavioral signal, can robustly differentiate human-authored from AI-assisted writing when used alongside text features. By expanding datasets and introducing paraphrasing, the authors benchmark gradient-boosted detectors (LightGBM, CatBoost) and a Siamese-TypeNet model, revealing that ML models excel in structured contexts while TypeNet excels in paraphrase scenarios. A deception threat model shows forged keystrokes can degrade performance, which the authors counter with adversarial training, achieving strong robustness. Across comparisons with DetectGPT, LLaMA-3.3-70B-Instruct, and human evaluators, purely text-based or subjective judgments perform near chance, highlighting the value of incorporating process-level data for academic integrity assessment. The study suggests a practical, multimodal approach to deter GenAI-assisted cheating while acknowledging limitations and ethical considerations, with future work focusing on larger, diverse datasets and additional behavioral cues.

Abstract

The rapid adoption of generative AI tools has intensified the challenge of maintaining academic integrity. Conventional plagiarism detectors, which rely on text-matching or text-intrinsic features, often fail to identify submissions that have been AI-assisted or paraphrased. To address this limitation, we introduce keystroke-dynamics-based detectors that analyze how, rather than what, a person writes to distinguish genuine from assisted writing. Building on our earlier study, which collected keystroke data from 40 participants and trained a modified TypeNet model to detect assisted text, we expanded the dataset by adding 90 new participants and introducing a paraphrasing-based plagiarism-detection mode. We then benchmarked two additional gradient-boosting classifiers, LightGBM and CatBoost, alongside TypeNet, and compared their performance with DetectGPT, LLaMA 3.3 70B Instruct, and the results of 44 human evaluators. To further assess and improve robustness, we proposed a deception-based threat model simulating forged keystrokes and applied adversarial training as a countermeasure. Results show that the machine learning models achieve F1 scores above 97% in structured settings, while TypeNet performs best in detecting paraphrasing, with an F1 score of 86.9%. In contrast, text-only detectors and human evaluators perform near-chance, demonstrating that keystroke dynamics provide a strong behavioral signal for identifying AI-assisted plagiarism and support the use of multimodal behavioral features for reliable academic integrity assessment.

Paper Structure

This paper contains 35 sections, 4 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Materials and methods
  4. Datasets
  5. SBU Dataset
  6. Buffalo Dataset
  7. IIITD-BU-Transcribed dataset
  8. IIITD-BU-Paraphrased dataset
  9. Deception datasets
  10. Design of plagiarism detectors
  11. Keystroke modeling and classification
  12. ML-based detectors training
  13. DL-based detector training
  14. Threat Modeling and Robustness Evaluation
  15. Threat model
  16. ...and 20 more sections

Figures (4)

  • Figure 1: The plagiarism detection framework shows the input keystrokes, their segmentation, featurization, training of the models, and their evaluations on the test keystrokes for their ability to distinguish between the bona fide and plagiarized responses.
  • Figure 2: Histogram of $F_1$ scores illustrating the performance of human evaluators in detecting plagiarism. From left to right $\rightarrow$ (a) Comparison between expert and non-expert evaluators, (b) Comparison between plagiarism styles: paraphrasing versus transcription, and (c) Comparison between evaluation granularity: response-level versus window-level.
  • Figure 3: TSNE plot for the attack dataset with original data IIIT-BU-Paraphrased.
  • Figure 4: Decision boundaries: LGBM model decision boundaries before and after adversarial training. We sample and plot 50 attack samples after scaling and observe the changes in the decision boundary induced by adversarial training.