RubiSCoT: A Framework for AI-Supported Academic Assessment

TL;DR

RubiSCoT addresses the challenge of inconsistent and time-consuming thesis evaluation in higher education by introducing an AI-supported framework that combines large language models, retrieval-augmented generation, and structured chain-of-thought prompting. The system implements a rubric-based, multi-dimensional assessment pipeline—from preliminary checks to content extraction, rubric scoring, and reporting—grounded in Design Science Research. Key contributions include a principled integration of rubrics, explainable intermediate reasoning, and external document retrieval to ensure alignment with academic standards. This approach promises improved consistency, scalability, and actionable feedback, with ongoing empirical validation planned across institutions.

Abstract

The evaluation of academic theses is a cornerstone of higher education, ensuring rigor and integrity. Traditional methods, though effective, are time-consuming and subject to evaluator variability. This paper presents RubiSCoT, an AI-supported framework designed to enhance thesis evaluation from proposal to final submission. Using advanced natural language processing techniques, including large language models, retrieval-augmented generation, and structured chain-of-thought prompting, RubiSCoT offers a consistent, scalable solution. The framework includes preliminary assessments, multidimensional assessments, content extraction, rubric-based scoring, and detailed reporting. We present the design and implementation of RubiSCoT, discussing its potential to optimize academic assessment processes through consistent, scalable, and transparent evaluation.

Figures (7)

• Figure 1: Example of Rubrics.
• Figure 2: RubiSCoT’s Key Components.
• Figure 3: Assessment by Group.
• Figure 4: Content Extraction.
• Figure 5: Visual Flow Diagrams of the Thesis’s Logical Progression.