Using LLMs to support assessment of student work in higher education: a viva voce simulator

TL;DR

The paper addresses the integrity of long-form assessment in higher education amid widespread LLM use and proposes a virtual viva voce simulator powered by LLMs to verify authorship. It details a practical approach in which an LLM acts as the examiner, engaging the student with open-ended, text-based questions about their submitted work and producing an auditable transcript for human judgment. A proof-of-concept using a Gemini-based web app demonstrates feasibility, including a sample transcript and a critical analysis of deployment challenges and risks. The work contributes a scalable, auditable framework for assessment in AI-rich environments, while acknowledging potential security, equity, and fairness concerns that require careful real-world considerations.

Abstract

One of the emergent challenges of student work submitted for assessment is the widespread use of large language models (LLMs) to support and even produce written work. This particularly affects subjects where long-form written work is a key part of assessment. We propose a novel approach to addressing this challenge, using LLMs themselves to support the assessment process. We have developed a proof-of-concept viva voce examination simulator, which accepts the student's written submission as input, generates an interactive series of questions from the LLM and answers from the student. The viva voce simulator is an interactive tool which asks questions which a human examiner might plausibly ask, and uses the student's answers to form a judgment about whether the submitted piece of work is likely to be the student's own work. The interaction transcript is provided to the human examiner to support their final judgment. We suggest theoretical and practical points which are critical to real-world deployment of such a tool.