Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EssayCBM: Rubric-Aligned Concept Bottleneck Models for Transparent Essay Grading

Kumar Satvik Chaudhary, Chengshuai Zhao, Fan Zhang, Yung Hin Tse, Garima Agrawal, Yuli Deng, Huan Liu

TL;DR

Automated essay grading often remains a black box despite competitive accuracy. The authors propose EssayCBM, a rubric-aligned Concept Bottleneck Model that first predicts eight rubric concepts $C=\\\ o c_1,\\dots, c_8\\$ from essays and then computes the final grade via $\\hat{Y} = h(C)$, providing a transparent trace from rubric criteria to score. The model is trained end-to-end with $\\mathcal{L}_{\\text{total}} = \\mathcal{L}_{\\text{grade}} + \\lambda \\mathcal{L}_{\\text{concept}}$ and supports multiple encoders (e.g., BERT-base, RoBERTa-base, GPT-2, BiLSTM). It is deployed as an open-source web application with a Streamlit frontend and FastAPI backend, offering real-time rubric-aligned feedback, confidence scores, and human-in-the-loop correction. The results show that EssayCBM matches black-box performance while delivering actionable, concept-level explanations that can enhance formative feedback and trust in automated assessment.

Abstract

Understanding how automated grading systems evaluate essays remains a significant challenge for educators and students, especially when large language models function as black boxes. We introduce EssayCBM, a rubric-aligned framework that prioritizes interpretability in essay assessment. Instead of predicting grades directly from text, EssayCBM evaluates eight writing concepts, such as Thesis Clarity and Evidence Use, through dedicated prediction heads on an encoder. These concept scores form a transparent bottleneck, and a lightweight network computes the final grade using only concepts. Instructors can adjust concept predictions and instantly view the updated grade, enabling accountable human-in-the-loop evaluation. EssayCBM matches black-box performance while offering actionable, concept-level feedback through an intuitive web interface.

EssayCBM: Rubric-Aligned Concept Bottleneck Models for Transparent Essay Grading

TL;DR

Automated essay grading often remains a black box despite competitive accuracy. The authors propose EssayCBM, a rubric-aligned Concept Bottleneck Model that first predicts eight rubric concepts from essays and then computes the final grade via , providing a transparent trace from rubric criteria to score. The model is trained end-to-end with and supports multiple encoders (e.g., BERT-base, RoBERTa-base, GPT-2, BiLSTM). It is deployed as an open-source web application with a Streamlit frontend and FastAPI backend, offering real-time rubric-aligned feedback, confidence scores, and human-in-the-loop correction. The results show that EssayCBM matches black-box performance while delivering actionable, concept-level explanations that can enhance formative feedback and trust in automated assessment.

Abstract

Understanding how automated grading systems evaluate essays remains a significant challenge for educators and students, especially when large language models function as black boxes. We introduce EssayCBM, a rubric-aligned framework that prioritizes interpretability in essay assessment. Instead of predicting grades directly from text, EssayCBM evaluates eight writing concepts, such as Thesis Clarity and Evidence Use, through dedicated prediction heads on an encoder. These concept scores form a transparent bottleneck, and a lightweight network computes the final grade using only concepts. Instructors can adjust concept predictions and instantly view the updated grade, enabling accountable human-in-the-loop evaluation. EssayCBM matches black-box performance while offering actionable, concept-level feedback through an intuitive web interface.
Paper Structure (11 sections, 5 equations, 3 figures, 1 table)

This paper contains 11 sections, 5 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Proposed EssayCBM Framework
  4. Essay Encoding and Concept Prediction
  5. Bottleneck Architecture
  6. Training and Inference
  7. EssayCBM Architecture and Application
  8. Frontend
  9. Backend
  10. Application Scenarios
  11. Conclusion

Figures (3)

  • Figure 1: Transparent concept-level grading in EssayCBM versus black-box AES systems.
  • Figure 2: Overview of EssayCBM Architecture. Essays are encoded to predict rubric-aligned concepts, which are aggregated into a final grade. The frontend provides real-time feedback with confidence scores and human-in-the-loop correction.
  • Figure 3: An example of the EssayCBM Streamlit frontend, where users input essays, select encoder models, and receive rubric-aligned concept scores with real-time feedback and overall grade.