Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable Multi-View Deep Networks Methodology for Experimental Physics

Nadav Schneider, Muriel Tzdaka, Galit Sturm, Guy Lazovski, Galit Bar, Gilad Oren, Raz Gvishi, Gal Oren

TL;DR

The paper addresses the challenge of explainability in multi-view deep learning for physics experiments. It introduces four architectures (CSV, SSG, PSG, CDV) and an EXPLAINer-based methodology to produce per-view explanations despite view-pooling. Applied to High Energy Density Physics foam-quality classification with five views per sample, the approach achieves $84\%$ accuracy and $93\%$ AUC with the SSG architecture, surpassing a CSV baseline. The work highlights a trade-off between performance and per-view explainability and suggests broader applicability to other scientific domains.

Abstract

Physical experiments often involve multiple imaging representations, such as X-ray scans and microscopic images. Deep learning models have been widely used for supervised analysis in these experiments. Combining different image representations is frequently required to analyze and make a decision properly. Consequently, multi-view data has emerged - datasets where each sample is described by views from different angles, sources, or modalities. These problems are addressed with the concept of multi-view learning. Understanding the decision-making process of deep learning models is essential for reliable and credible analysis. Hence, many explainability methods have been devised recently. Nonetheless, there is a lack of proper explainability in multi-view models, which are challenging to explain due to their architectures. In this paper, we suggest different multi-view architectures for the vision domain, each suited to another problem, and we also present a methodology for explaining these models. To demonstrate the effectiveness of our methodology, we focus on the domain of High Energy Density Physics (HEDP) experiments, where multiple imaging representations are used to assess the quality of foam samples. We apply our methodology to classify the foam samples quality using the suggested multi-view architectures. Through experimental results, we showcase the improvement of accurate architecture choice on both accuracy - 78% to 84% and AUC - 83% to 93% and present a trade-off between performance and explainability. Specifically, we demonstrate that our approach enables the explanation of individual one-view models, providing insights into the decision-making process of each view. This understanding enhances the interpretability of the overall multi-view model. The sources of this work are available at: https://github.com/Scientific-Computing-Lab-NRCN/Multi-View-Explainability.

Explainable Multi-View Deep Networks Methodology for Experimental Physics

TL;DR

The paper addresses the challenge of explainability in multi-view deep learning for physics experiments. It introduces four architectures (CSV, SSG, PSG, CDV) and an EXPLAINer-based methodology to produce per-view explanations despite view-pooling. Applied to High Energy Density Physics foam-quality classification with five views per sample, the approach achieves accuracy and AUC with the SSG architecture, surpassing a CSV baseline. The work highlights a trade-off between performance and per-view explainability and suggests broader applicability to other scientific domains.

Abstract

Physical experiments often involve multiple imaging representations, such as X-ray scans and microscopic images. Deep learning models have been widely used for supervised analysis in these experiments. Combining different image representations is frequently required to analyze and make a decision properly. Consequently, multi-view data has emerged - datasets where each sample is described by views from different angles, sources, or modalities. These problems are addressed with the concept of multi-view learning. Understanding the decision-making process of deep learning models is essential for reliable and credible analysis. Hence, many explainability methods have been devised recently. Nonetheless, there is a lack of proper explainability in multi-view models, which are challenging to explain due to their architectures. In this paper, we suggest different multi-view architectures for the vision domain, each suited to another problem, and we also present a methodology for explaining these models. To demonstrate the effectiveness of our methodology, we focus on the domain of High Energy Density Physics (HEDP) experiments, where multiple imaging representations are used to assess the quality of foam samples. We apply our methodology to classify the foam samples quality using the suggested multi-view architectures. Through experimental results, we showcase the improvement of accurate architecture choice on both accuracy - 78% to 84% and AUC - 83% to 93% and present a trade-off between performance and explainability. Specifically, we demonstrate that our approach enables the explanation of individual one-view models, providing insights into the decision-making process of each view. This understanding enhances the interpretability of the overall multi-view model. The sources of this work are available at: https://github.com/Scientific-Computing-Lab-NRCN/Multi-View-Explainability.
Paper Structure (14 sections, 6 figures, 1 table, 1 algorithm)

This paper contains 14 sections, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Multi-View Learning in Physics
  3. Multi-view Model Explainability
  4. Contribution
  5. Multi-View Architectures
  6. Multi-View Explainability
  7. Problem Definition
  8. Suggested Solution
  9. High Energy Density Physics Use Case
  10. Dataset
  11. Experimental Results
  12. Evaluation
  13. Explainability
  14. Conclusions and Future Work

Figures (6)

  • Figure 1: Multi-View model gets as an input M+N different views from a certain image diagnostic. In this example, foam quality is assessed for HEDP experiments. There are two visually different sub-group views. Then, the model outputs a classification and the explanation behind it, explainability is both local and global.
  • Figure 2: The four different suggested multi-view architectures which are suited to different problems: (a) assumes views are similar, (b) assumes visually similar sub-groups, (c) assumes similar sub-groups but also different in a way, and (d) assumes views are completely different. EXPLAINer content is demonstrated in \ref{['fig:explainer']}. Note that CNN can be replaced with any other feature extractor.
  • Figure 3: Multi-View Explainability
  • Figure 4: Demonstrating the Multi-View architectures correspondingly to \ref{['fig:multi_view_architectures']} on a real-life physical problem. In this case, there are 5 different views and 2 visually different sub-groups in the foam quality assessment for the HEDP experiments domain. (a) refers to all different views as visually similar, which is wrong. (b) does the correct splitting between the sub-groups, (c) can be correct to some degree and (d) has redundant feature extractors assuming each view is completely different than the other. EXPLAINer is connected to each feature extractor to explain its decision.
  • Figure 5: Multi-View local explainability with LIME. Areas of interest are marked with orange in each view. Cracks in the profiles can be watched, indicating a contribution towards defective foam decision. On the other hand, empty areas are marked in the top-bottom views, contributing towards a normal decision.
  • ...and 1 more figures