Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Comgra: A Tool for Analyzing and Debugging Neural Networks

Florian Dietz, Sophie Fellenz, Dietrich Klakow, Marius Kloft

TL;DR

Comgra addresses the challenge of inspecting large, complex neural networks by providing a PyTorch-compatible tool with a GUI that logs internal activations and gradients. The approach combines dynamic, flexible logging with a dependency-graph view to enable rapid, multi-angle inspection of tensors, training dynamics, and interventions, while maintaining a manageable memory footprint. Key contributions include the GUI architecture with selectors, a customizable dependency graph, and workflow-oriented use cases that demonstrate debugging, architecture exploration, and mechanistic interpretability. The work enables faster hypothesis testing and targeted debugging without rerunning extensive training runs, improving practicality for researchers and practitioners alike.

Abstract

Neural Networks are notoriously difficult to inspect. We introduce comgra, an open source python library for use with PyTorch. Comgra extracts data about the internal activations of a model and organizes it in a GUI (graphical user interface). It can show both summary statistics and individual data points, compare early and late stages of training, focus on individual samples of interest, and visualize the flow of the gradient through the network. This makes it possible to inspect the model's behavior from many different angles and save time by rapidly testing different hypotheses without having to rerun it. Comgra has applications for debugging, neural architecture design, and mechanistic interpretability. We publish our library through Python Package Index (PyPI) and provide code, documentation, and tutorials at https://github.com/FlorianDietz/comgra.

Comgra: A Tool for Analyzing and Debugging Neural Networks

TL;DR

Comgra addresses the challenge of inspecting large, complex neural networks by providing a PyTorch-compatible tool with a GUI that logs internal activations and gradients. The approach combines dynamic, flexible logging with a dependency-graph view to enable rapid, multi-angle inspection of tensors, training dynamics, and interventions, while maintaining a manageable memory footprint. Key contributions include the GUI architecture with selectors, a customizable dependency graph, and workflow-oriented use cases that demonstrate debugging, architecture exploration, and mechanistic interpretability. The work enables faster hypothesis testing and targeted debugging without rerunning extensive training runs, improving practicality for researchers and practitioners alike.

Abstract

Neural Networks are notoriously difficult to inspect. We introduce comgra, an open source python library for use with PyTorch. Comgra extracts data about the internal activations of a model and organizes it in a GUI (graphical user interface). It can show both summary statistics and individual data points, compare early and late stages of training, focus on individual samples of interest, and visualize the flow of the gradient through the network. This makes it possible to inspect the model's behavior from many different angles and save time by rapidly testing different hypotheses without having to rerun it. Comgra has applications for debugging, neural architecture design, and mechanistic interpretability. We publish our library through Python Package Index (PyPI) and provide code, documentation, and tutorials at https://github.com/FlorianDietz/comgra.
Paper Structure (10 sections, 1 figure)

This paper contains 10 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Existing Tools for Neural Network Analysis
  3. Comgra
  4. Graphical User Interface
  5. Dynamic Logging
  6. Usecases
  7. Future Work
  8. Conclusion
  9. Author Contributions
  10. Acknowledgements

Figures (1)

  • Figure 1: Comgra's GUI. Top: Selectors for the version of the selected tensor and aspects you want to inspect. Middle: The dependency graph, in which you can click on a tensor to select it. Bottom: Summary statistics as well as raw values for the selected tensor under the selected criteria.