Diverse Explanations From Data-Driven and Domain-Driven Perspectives in the Physical Sciences

TL;DR

This analysis underscores the importance of considering multiple perspectives when interpreting ML models in scientific contexts and highlights the critical need for scientists to maintain control over the interpretation process, balancing data-driven insights with domain expertise to meet specific scientific needs.

Abstract

Machine learning methods have been remarkably successful in material science, providing novel scientific insights, guiding future laboratory experiments, and accelerating materials discovery. Despite the promising performance of these models, understanding the decisions they make is also essential to ensure the scientific value of their outcomes. However, there is a recent and ongoing debate about the diversity of explanations, which potentially leads to scientific inconsistency. This Perspective explores the sources and implications of these diverse explanations in ML applications for physical sciences. Through three case studies in materials science and molecular property prediction, we examine how different models, explanation methods, levels of feature attribution, and stakeholder needs can result in varying interpretations of ML outputs. Our analysis underscores the importance of considering multiple perspectives when interpreting ML models in scientific contexts and highlights the critical need for scientists to maintain control over the interpretation process, balancing data-driven insights with domain expertise to meet specific scientific needs. By fostering a comprehensive understanding of these inconsistencies, we aim to contribute to the responsible integration of eXplainable Artificial Intelligence (XAI) into physical sciences and improve the trustworthiness of ML applications in scientific discovery.

Figures (8)

• Figure 1: The potential conflicts from data-driven and domain-driven explanations in the decision-making process involving well-trained ML models. The dashed line denotes the conventional pipeline of XAI in the scientific domain, where stakeholders utilise ML models and retrieve data-driven explanations to analyse results. In practice, it is common that data-driven explanations conflict with domain knowledge, misleading researchers, and requiring a new approach.
• Figure 2: Visualisation of four sources of inconsistent explanations.
• Figure 3: The average contribution of all elements to bulk modulus predictions, computed from the AFLOW bulk modulus dataset. (a) MAE scores of Roost, CrabNet, one-hot encoded CrabNet (HotCrab), ElemNet, and MLP on the held-out test datasets, compared with the random forest (RF) baseline for the property. (b) Figure reprinted from wang2021compositionally under the CC BY 4.0 license commons2013creative. (c) First-order feature attribution calculated based on the well-trained MLP. The lighter-coloured elements in the periodic table contribute more towards a compound’s bulk modulus value.
• Figure 4: (a) Feature importance rankings of metallic nanoparticles from accurate models, including XGBoost, MLP, and RF, and (b) feature importance rankings (MLP) from different well-established explanation methods including PI, SHAP and IG. The x-axis displays features ordered by their importance ranking from RF, which serves as a baseline. Other rankings are plotted according to this order. The y-axis denotes the importance score, normalised between 0 and 1.
• Figure 5: Feature importance rankings of metallic nanoparticles fractal dimension predictions from four stakeholders (scenarios). Features ordered by their importance rankings (left to right), serving as a baseline. Other rankings are plotted according to this order. The $y$-axis denotes the importance score, normalised between 0 and 1.