Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decide: Knowledge-Based Version Incompatibility Detection in Deep Learning Stacks

Zihan Zhou, Zhongkai Zhao, Bonan Kou, Tianyi Zhang

TL;DR

This work addresses the challenge of DL stack version incompatibilities by harvesting knowledge from Stack Overflow to build a large knowledge graph of component (in)compatibilities. It introduces Decide, an interactive web tool featuring a compatibility visualizer, an information panel with source-cited posts, a flexible search interface, and a statistical panel, underpinned by a QA-based inference pipeline using UnifiedQA. The knowledge graph comprises 2,376 relations among 48 DL components collected from 355K posts, achieving high precision and useful recall in detecting incompatibilities, with a reported 83.7% accuracy on a sampled evaluation and 91.7% precision in a GitHub-based benchmark. The results demonstrate the potential of crowd-sourced version knowledge to reduce DL deployment failures and guide developers in reusing or reproducing models and pipelines, while acknowledging limitations and future expansion to broader data sources and components.

Abstract

Version incompatibility issues are prevalent when reusing or reproducing deep learning (DL) models and applications. Compared with official API documentation, which is often incomplete or out-of-date, Stack Overflow (SO) discussions possess a wealth of version knowledge that has not been explored by previous approaches. To bridge this gap, we present Decide, a web-based visualization of a knowledge graph that contains 2,376 version knowledge extracted from SO discussions. As an interactive tool, Decide allows users to easily check whether two libraries are compatible and explore compatibility knowledge of certain DL stack components with or without the version specified. A video demonstrating the usage of Decide is available at https://youtu.be/wqPxF2ZaZo0.

Decide: Knowledge-Based Version Incompatibility Detection in Deep Learning Stacks

TL;DR

This work addresses the challenge of DL stack version incompatibilities by harvesting knowledge from Stack Overflow to build a large knowledge graph of component (in)compatibilities. It introduces Decide, an interactive web tool featuring a compatibility visualizer, an information panel with source-cited posts, a flexible search interface, and a statistical panel, underpinned by a QA-based inference pipeline using UnifiedQA. The knowledge graph comprises 2,376 relations among 48 DL components collected from 355K posts, achieving high precision and useful recall in detecting incompatibilities, with a reported 83.7% accuracy on a sampled evaluation and 91.7% precision in a GitHub-based benchmark. The results demonstrate the potential of crowd-sourced version knowledge to reduce DL deployment failures and guide developers in reusing or reproducing models and pipelines, while acknowledging limitations and future expansion to broader data sources and components.

Abstract

Version incompatibility issues are prevalent when reusing or reproducing deep learning (DL) models and applications. Compared with official API documentation, which is often incomplete or out-of-date, Stack Overflow (SO) discussions possess a wealth of version knowledge that has not been explored by previous approaches. To bridge this gap, we present Decide, a web-based visualization of a knowledge graph that contains 2,376 version knowledge extracted from SO discussions. As an interactive tool, Decide allows users to easily check whether two libraries are compatible and explore compatibility knowledge of certain DL stack components with or without the version specified. A video demonstrating the usage of Decide is available at https://youtu.be/wqPxF2ZaZo0.
Paper Structure (15 sections, 1 equation, 3 figures, 3 tables)

This paper contains 15 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Knowledge Graph Construction
  3. Data Collection and Filtering
  4. DL Stack Component Recognition
  5. Compatibility Inference via a Pre-trained QA Model
  6. Knowledge Graph Visualization
  7. The Compatibility Visualizer
  8. The Information Panel
  9. DL Library Statistics
  10. Relationship Details
  11. The Search Bar
  12. The Statistical Panel
  13. Evaluation
  14. Discussion
  15. Conclusion

Figures (3)

  • Figure 1: An overview of the knowledge graph construction and incompatibility detection process
  • Figure 2: QA examples for version compatibility inference
  • Figure 3: Decide, an interactive knowledge-based tool for detecting and identifying version (in)compatibility in deep learning stacks. (A) The Compatibility Visualizer. (B) The Information Panel. (C) The Search Bar. (D) The Statistical Panel.