Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring the Relationship Between Feature Attribution Methods and Model Performance

Priscylla Silva, Claudio T. Silva, Luis Gustavo Nonato

TL;DR

The paper tackles the problem of explainability in educational predictions by analyzing whether higher predictive performance correlates with stronger consensus among nine feature attribution methods. It frames student-success prediction as a binary task and uses nine attribution techniques, four (dis)agreement metrics, and two real-world datasets with intermediate-epoch model snapshots to quantify how explanation agreement evolves with model quality. The study finds a very strong Spearman correlation between AUC and agreement across methods, indicating that better-performing models yield more consistent explanations, with practical implications for selecting models and interpreting predictions in education. These results highlight an intrinsic link between model performance and interpretability, supporting the use of high-AUC models to improve the reliability of explanation-driven decisions in educational settings.

Abstract

Machine learning and deep learning models are pivotal in educational contexts, particularly in predicting student success. Despite their widespread application, a significant gap persists in comprehending the factors influencing these models' predictions, especially in explainability within education. This work addresses this gap by employing nine distinct explanation methods and conducting a comprehensive analysis to explore the correlation between the agreement among these methods in generating explanations and the predictive model's performance. Applying Spearman's correlation, our findings reveal a very strong correlation between the model's performance and the agreement level observed among the explanation methods.

Exploring the Relationship Between Feature Attribution Methods and Model Performance

TL;DR

The paper tackles the problem of explainability in educational predictions by analyzing whether higher predictive performance correlates with stronger consensus among nine feature attribution methods. It frames student-success prediction as a binary task and uses nine attribution techniques, four (dis)agreement metrics, and two real-world datasets with intermediate-epoch model snapshots to quantify how explanation agreement evolves with model quality. The study finds a very strong Spearman correlation between AUC and agreement across methods, indicating that better-performing models yield more consistent explanations, with practical implications for selecting models and interpreting predictions in education. These results highlight an intrinsic link between model performance and interpretability, supporting the use of high-AUC models to improve the reliability of explanation-driven decisions in educational settings.

Abstract

Machine learning and deep learning models are pivotal in educational contexts, particularly in predicting student success. Despite their widespread application, a significant gap persists in comprehending the factors influencing these models' predictions, especially in explainability within education. This work addresses this gap by employing nine distinct explanation methods and conducting a comprehensive analysis to explore the correlation between the agreement among these methods in generating explanations and the predictive model's performance. Applying Spearman's correlation, our findings reveal a very strong correlation between the model's performance and the agreement level observed among the explanation methods.
Paper Structure (15 sections, 4 equations, 6 figures, 1 table)

This paper contains 15 sections, 4 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Formulation
  4. Experimental Setup
  5. Model Performance in Intermediate Epochs
  6. (Dis)agreement measurement
  7. Correlation Analysis
  8. Results and Discussions
  9. Discussion and Conclusion
  10. Disagreement Metrics
  11. Explanation Methods
  12. Datasets
  13. Data Distribution of the (dis)agreement score
  14. Example of (dis)agreement between the pairs of methods
  15. AUC vs Disagreement Level

Figures (6)

  • Figure 1: Correlation between Model Performance (AUC) and (Dis)agreement Metrics for Models Trained on the Introductory Programming Course Dataset.
  • Figure 2: Correlation between Model Performance (AUC) and (Dis)agreement Metrics for Models Trained on Amrieh_2015's Dataset.
  • Figure 3: Boxplots illustrating the distribution of the disagreement level score by FA metric.
  • Figure 4: Heatmap illustrating the (dis)agreement levels between explanation methods.
  • Figure 5: Correlation between Model Performance (AUC) and Disagreement Metrics for Models Trained on the Introductory Programming Course Dataset.
  • ...and 1 more figures