Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BERT-Based Approach for Automating Course Articulation Matrix Construction with Explainable AI

Natenaile Asmamaw Shiferaw, Simpenzwe Honore Leandre, Aman Sinha, Dillip Rout

TL;DR

The potential of utilizing transfer learning with BERT-based models for the automated generation of CAMs, offering high performance and interpretability in educational outcome assessment is demonstrated.

Abstract

Course Outcome (CO) and Program Outcome (PO)/Program-Specific Outcome (PSO) alignment is a crucial task for ensuring curriculum coherence and assessing educational effectiveness. The construction of a Course Articulation Matrix (CAM), which quantifies the relationship between COs and POs/PSOs, typically involves assigning numerical values (0, 1, 2, 3) to represent the degree of alignment. In this study, We experiment with four models from the BERT family: BERT Base, DistilBERT, ALBERT, and RoBERTa, and use multiclass classification to assess the alignment between CO and PO/PSO pairs. We first evaluate traditional machine learning classifiers, such as Decision Tree, Random Forest, and XGBoost, and then apply transfer learning to evaluate the performance of the pretrained BERT models. To enhance model interpretability, we apply Explainable AI technique, specifically Local Interpretable Model-agnostic Explanations (LIME), to provide transparency into the decision-making process. Our system achieves accuracy, precision, recall, and F1-score values of 98.66%, 98.67%, 98.66%, and 98.66%, respectively. This work demonstrates the potential of utilizing transfer learning with BERT-based models for the automated generation of CAMs, offering high performance and interpretability in educational outcome assessment.

BERT-Based Approach for Automating Course Articulation Matrix Construction with Explainable AI

TL;DR

The potential of utilizing transfer learning with BERT-based models for the automated generation of CAMs, offering high performance and interpretability in educational outcome assessment is demonstrated.

Abstract

Course Outcome (CO) and Program Outcome (PO)/Program-Specific Outcome (PSO) alignment is a crucial task for ensuring curriculum coherence and assessing educational effectiveness. The construction of a Course Articulation Matrix (CAM), which quantifies the relationship between COs and POs/PSOs, typically involves assigning numerical values (0, 1, 2, 3) to represent the degree of alignment. In this study, We experiment with four models from the BERT family: BERT Base, DistilBERT, ALBERT, and RoBERTa, and use multiclass classification to assess the alignment between CO and PO/PSO pairs. We first evaluate traditional machine learning classifiers, such as Decision Tree, Random Forest, and XGBoost, and then apply transfer learning to evaluate the performance of the pretrained BERT models. To enhance model interpretability, we apply Explainable AI technique, specifically Local Interpretable Model-agnostic Explanations (LIME), to provide transparency into the decision-making process. Our system achieves accuracy, precision, recall, and F1-score values of 98.66%, 98.67%, 98.66%, and 98.66%, respectively. This work demonstrates the potential of utilizing transfer learning with BERT-based models for the automated generation of CAMs, offering high performance and interpretability in educational outcome assessment.

Paper Structure

This paper contains 31 sections, 11 equations, 9 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Advancements in Curriculum Mapping
  4. Evolution of Text Classification Models
  5. Methodology
  6. Data Collection and Preprocessing
  7. Dataset Preparation
  8. Data Augmentation
  9. Data Preprocessing
  10. Model Description
  11. Machine Learning Classifiers
  12. Pre-trained Transformer Models
  13. Transfer Learning
  14. Model optimization and training
  15. Adam Optimizer
  16. ...and 16 more sections

Figures (9)

  • Figure 1: Workflow of the proposed model using transfer learning, from data collection and augmentation to BERT fine-tuning and model interpretation.
  • Figure 2: Visualization of ROC Curves for Machine Learning Classifiers in the Construction Task.
  • Figure 3: Visualization of Accuracy and Loss Graphs with Respect to Epochs for the Construction Task.
  • Figure 4: Comparison of training times for BERT-base, DistilBERT, ALBERT, and RoBERTa with colored bars indicating each model.
  • Figure 5: Confusion Matrix for -/ Alignment Score Prediction on the Validation Dataset.
  • ...and 4 more figures