Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learning

Sarah Segel, Helena Graf, Edward Bergman, Kristina Thieme, Marcel Wever, Alexander Tornede, Frank Hutter, Marius Lindauer

TL;DR

The paper introduces DeepCAVE, a visualization and analysis tool designed to improve interpretability and debugging of hyperparameter optimization in AutoML. Built on Python and Dash, it uses run objects and modular converters and plugins to support multiple optimizers, NAS, and multi-objective/multi-fidelity scenarios. The system enables detailed, configurable analyses across runs, including configuration footprints, Pareto fronts, and hyperparameter importances, with asynchronous execution to maintain responsiveness. By facilitating cross-run comparisons and exportable visuals, DeepCAVE aims to enhance reproducibility and foster development of more robust AutoML methods.

Abstract

Hyperparameter optimization (HPO), as a central paradigm of AutoML, is crucial for leveraging the full potential of machine learning (ML) models; yet its complexity poses challenges in understanding and debugging the optimization process. We present DeepCAVE, a tool for interactive visualization and analysis, providing insights into HPO. Through an interactive dashboard, researchers, data scientists, and ML engineers can explore various aspects of the HPO process and identify issues, untouched potentials, and new insights about the ML model being tuned. By empowering users with actionable insights, DeepCAVE contributes to the interpretability of HPO and ML on a design level and aims to foster the development of more robust and efficient methodologies in the future.

DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learning

TL;DR

The paper introduces DeepCAVE, a visualization and analysis tool designed to improve interpretability and debugging of hyperparameter optimization in AutoML. Built on Python and Dash, it uses run objects and modular converters and plugins to support multiple optimizers, NAS, and multi-objective/multi-fidelity scenarios. The system enables detailed, configurable analyses across runs, including configuration footprints, Pareto fronts, and hyperparameter importances, with asynchronous execution to maintain responsiveness. By facilitating cross-run comparisons and exportable visuals, DeepCAVE aims to enhance reproducibility and foster development of more robust AutoML methods.

Abstract

Hyperparameter optimization (HPO), as a central paradigm of AutoML, is crucial for leveraging the full potential of machine learning (ML) models; yet its complexity poses challenges in understanding and debugging the optimization process. We present DeepCAVE, a tool for interactive visualization and analysis, providing insights into HPO. Through an interactive dashboard, researchers, data scientists, and ML engineers can explore various aspects of the HPO process and identify issues, untouched potentials, and new insights about the ML model being tuned. By empowering users with actionable insights, DeepCAVE contributes to the interpretability of HPO and ML on a design level and aims to foster the development of more robust and efficient methodologies in the future.

Paper Structure

This paper contains 6 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Features and Usage
  3. Converters
  4. Analysis Plugins
  5. Existing Tools for Explainable AutoML
  6. Conclusion

Figures (2)

  • Figure 1: Overview of DeepCAVE's components and their functionalities.
  • Figure 2: Examples of plots produced via DeepCAVE's plugins. A mouseover allows obtaining additional information, and clicking on single configurations in the plot opens the Configurations plugin, providing details regarding that configuration.