Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physics-Inspired Interpretability Of Machine Learning Models

Maximilian P Niroomand, David J Wales

TL;DR

A novel approach to identify relevant features of the input data, inspired by methods from the energy landscapes field, developed in the physical sciences, is proposed, which can identify the drivers of model decision making.

Abstract

The ability to explain decisions made by machine learning models remains one of the most significant hurdles towards widespread adoption of AI in highly sensitive areas such as medicine, cybersecurity or autonomous driving. Great interest exists in understanding which features of the input data prompt model decision making. In this contribution, we propose a novel approach to identify relevant features of the input data, inspired by methods from the energy landscapes field, developed in the physical sciences. By identifying conserved weights within groups of minima of the loss landscapes, we can identify the drivers of model decision making. Analogues to this idea exist in the molecular sciences, where coordinate invariants or order parameters are employed to identify critical features of a molecule. However, no such approach exists for machine learning loss landscapes. We will demonstrate the applicability of energy landscape methods to machine learning models and give examples, both synthetic and from the real world, for how these methods can help to make models more interpretable.

Physics-Inspired Interpretability Of Machine Learning Models

TL;DR

A novel approach to identify relevant features of the input data, inspired by methods from the energy landscapes field, developed in the physical sciences, is proposed, which can identify the drivers of model decision making.

Abstract

The ability to explain decisions made by machine learning models remains one of the most significant hurdles towards widespread adoption of AI in highly sensitive areas such as medicine, cybersecurity or autonomous driving. Great interest exists in understanding which features of the input data prompt model decision making. In this contribution, we propose a novel approach to identify relevant features of the input data, inspired by methods from the energy landscapes field, developed in the physical sciences. By identifying conserved weights within groups of minima of the loss landscapes, we can identify the drivers of model decision making. Analogues to this idea exist in the molecular sciences, where coordinate invariants or order parameters are employed to identify critical features of a molecule. However, no such approach exists for machine learning loss landscapes. We will demonstrate the applicability of energy landscape methods to machine learning models and give examples, both synthetic and from the real world, for how these methods can help to make models more interpretable.
Paper Structure (9 sections, 2 figures)

This paper contains 9 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Energy landscapes
  3. Related work
  4. Energy landscapes methods
  5. Landscape visualisation
  6. Experiments
  7. Permutational invariance groups
  8. Discussion and conclusions
  9. Acknowledgements

Figures (2)

  • Figure 1: Disconnectivity graph for the checkerboard dataset. The conserved weights for a specific local minimum are highlighted in the respective colour for the chosen examples.
  • Figure 2: Disconnectivity graph for credit card data. Group 25_7 in red includes the global minimum. Coloured edges indicate that for all minima in the specific group, these particular weights are conserved, i.e. have a standard deviation $< n$ which has been set to $n = 0.01$ here.