Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Keystroke Dynamics Against Academic Dishonesty in the Age of LLMs

Debnath Kundu, Atharva Mehta, Rajesh Kumar, Naman Lal, Avinash Anand, Apoorv Singh, Rajiv Ratn Shah

TL;DR

A keystroke dynamics-based method to differentiate between bona fide and assisted writing within academic contexts is proposed and the findings highlight significant differences in keystroke dynamics between genuine and assisted writing.

Abstract

The transition to online examinations and assignments raises significant concerns about academic integrity. Traditional plagiarism detection systems often struggle to identify instances of intelligent cheating, particularly when students utilize advanced generative AI tools to craft their responses. This study proposes a keystroke dynamics-based method to differentiate between bona fide and assisted writing within academic contexts. To facilitate this, a dataset was developed to capture the keystroke patterns of individuals engaged in writing tasks, both with and without the assistance of generative AI. The detector, trained using a modified TypeNet architecture, achieved accuracies ranging from 74.98% to 85.72% in condition-specific scenarios and from 52.24% to 80.54% in condition-agnostic scenarios. The findings highlight significant differences in keystroke dynamics between genuine and assisted writing. The outcomes of this study enhance our understanding of how users interact with generative AI and have implications for improving the reliability of digital educational platforms.

Keystroke Dynamics Against Academic Dishonesty in the Age of LLMs

TL;DR

A keystroke dynamics-based method to differentiate between bona fide and assisted writing within academic contexts is proposed and the findings highlight significant differences in keystroke dynamics between genuine and assisted writing.

Abstract

The transition to online examinations and assignments raises significant concerns about academic integrity. Traditional plagiarism detection systems often struggle to identify instances of intelligent cheating, particularly when students utilize advanced generative AI tools to craft their responses. This study proposes a keystroke dynamics-based method to differentiate between bona fide and assisted writing within academic contexts. To facilitate this, a dataset was developed to capture the keystroke patterns of individuals engaged in writing tasks, both with and without the assistance of generative AI. The detector, trained using a modified TypeNet architecture, achieved accuracies ranging from 74.98% to 85.72% in condition-specific scenarios and from 52.24% to 80.54% in condition-agnostic scenarios. The findings highlight significant differences in keystroke dynamics between genuine and assisted writing. The outcomes of this study enhance our understanding of how users interact with generative AI and have implications for improving the reliability of digital educational platforms.
Paper Structure (12 sections, 3 figures, 3 tables)

This paper contains 12 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Design of experiment
  4. Datasets
  5. Training detection models
  6. Evaluation scenarios
  7. Evaluation metrics
  8. Results and discussion
  9. Discussion
  10. Limitations
  11. Conclusions and future work
  12. Acknowledgment

Figures (3)

  • Figure 1: The framework of existing plagiarism detectors such as Turnitin and Urkund, which use content-based word-for-word similarities alzahrani2011understandingrao2008plagiarism.
  • Figure 2: The cognitive model of the writing process includes four stages: Proposer, where ideas are generated and tasks prepared, marked by initial pauses and pauses at sentence boundaries; Translator, involving the fluent conversion of ideas into language, measured by uninterrupted text production sequences; Transcriber, focusing on orthographic proficiency and motor skills, evident in pauses within words and immediate spelling corrections; and Evaluator, which entails editing and reviewing, identifiable by jumps to different text parts for extensive edits zhang2021using.
  • Figure 3: Illustrating the proposed detection framework that transforms training and testing data for use in training the detection models, which then contribute to deciding whether the output is bonafide or assisted.