Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Surveillance: Live-Update Visualization of Latent Training Dynamics

Xianglin Yang, Jin Song Dong

TL;DR

The paper tackles the problem of real-time auditing of neural network training by visualizing evolving latent representations. It introduces SentryCam, a live-update visualization framework built on a dynamic spatio-temporal graph, density-guided sampling, and a GN-BN hybrid projection to produce faithful 2D representations with minimal latency. The authors provide theory linking sampling density to topology preservation via a relative-density metric and demonstrate a topology tipping point to guide practical sampling decisions, along with extensive experiments showing efficiency, high-quality visuals, and timely alerts for training failures. The approach yields actionable, early warnings and supports continual learning analyses, offering a practical tool for proactive model development and auditing with open-source availability.

Abstract

Monitoring the inner state of deep neural networks is essential for auditing the learning process and enabling timely interventions. While conventional metrics like validation loss offer a surface-level view of performance, the evolution of a model's hidden representations provides a deeper, complementary window into its internal dynamics. However, the literature lacks a real-time tool to monitor these crucial internal states. To address this, we introduce SentryCam, a live-update visualization framework that tracks the progression of hidden representations throughout training. SentryCam produces high-fidelity visualizations of the evolving representation space with minimal latency, serving as a powerful dashboard for understanding how a model learns. We quantitatively validate the faithfulness of SentryCam's visualizations across diverse datasets and architectures (ResNet, ViT). Furthermore, we demonstrate SentryCam's practical utility for model auditing through a case study on training instability. We designed an automated auditing system with geometry-based alerts that successfully identified impending model failure up to 7 epochs earlier than was evident from the validation loss curve. SentryCam's flexible framework is easily adaptable, supporting both the exploratory analysis and proactive auditing essential for robust model development. The code is available at https://github.com/xianglinyang/SentryCam.

Neural Surveillance: Live-Update Visualization of Latent Training Dynamics

TL;DR

The paper tackles the problem of real-time auditing of neural network training by visualizing evolving latent representations. It introduces SentryCam, a live-update visualization framework built on a dynamic spatio-temporal graph, density-guided sampling, and a GN-BN hybrid projection to produce faithful 2D representations with minimal latency. The authors provide theory linking sampling density to topology preservation via a relative-density metric and demonstrate a topology tipping point to guide practical sampling decisions, along with extensive experiments showing efficiency, high-quality visuals, and timely alerts for training failures. The approach yields actionable, early warnings and supports continual learning analyses, offering a practical tool for proactive model development and auditing with open-source availability.

Abstract

Monitoring the inner state of deep neural networks is essential for auditing the learning process and enabling timely interventions. While conventional metrics like validation loss offer a surface-level view of performance, the evolution of a model's hidden representations provides a deeper, complementary window into its internal dynamics. However, the literature lacks a real-time tool to monitor these crucial internal states. To address this, we introduce SentryCam, a live-update visualization framework that tracks the progression of hidden representations throughout training. SentryCam produces high-fidelity visualizations of the evolving representation space with minimal latency, serving as a powerful dashboard for understanding how a model learns. We quantitatively validate the faithfulness of SentryCam's visualizations across diverse datasets and architectures (ResNet, ViT). Furthermore, we demonstrate SentryCam's practical utility for model auditing through a case study on training instability. We designed an automated auditing system with geometry-based alerts that successfully identified impending model failure up to 7 epochs earlier than was evident from the validation loss curve. SentryCam's flexible framework is easily adaptable, supporting both the exploratory analysis and proactive auditing essential for robust model development. The code is available at https://github.com/xianglinyang/SentryCam.
Paper Structure (46 sections, 4 theorems, 7 equations, 15 figures, 5 tables, 1 algorithm)

This paper contains 46 sections, 4 theorems, 7 equations, 15 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. The Challenge of Live Latent Visualization
  3. Method
  4. Notations
  5. Dynamic Graph Construction
  6. Stage ❷-❸: Working Memory Construction.
  7. Stable Projection Learning
  8. Topology Preservation Under Sampling
  9. Guarantee of Topology Preservation via Relative Density Metric
  10. The Topology Tipping Point
  11. Experiments
  12. Experimental Setup
  13. RQ1: Efficiency
  14. RQ2: Visualization Quality
  15. RQ3: Practical Utility for Real-Time Auditing
  16. ...and 31 more sections

Key Result

Theorem 1

There exists a universal constant $c\in(0,1)$ such that if $d_H(S,\mathcal{M})=\varepsilon(S)<c\,\tau$, then there is a non-empty interval of radii $\alpha$ for which the Čech complex $\check{C}_\alpha(S)$ is homotopy equivalent to $\mathcal{M}$. Moreover, the Vietoris--Rips complex $R_\rho(S)$ is a

Figures (15)

  • Figure 1: Overview of SentryCam. First, as the subject model trains (stage ❶), our framework intercepts its hidden representations at each checkpoint (stage ❷). Next, we assemble a working memory of nodes, comprising the current representations and a curated set of historical ones. From this memory, we construct a composite graph that encodes both spatial and temporal relationships (stage ❸). To ensure real-time performance, this graph is immediately downsampled using our novel density-guided algorithm (stage ❹). Finally, a specialized autoencoder projects the resulting graph into a low-dimensional space to produce the final interpretable visualization (stage ❺).
  • Figure 2: The role of sample density in revealing underlying data structure. (a) The ground truth distribution of three distinct classes. (b) A sufficiently dense sample allows for the faithful reconstruction of the three-cluster structure. (c) A sparse sample provides insufficient information, causing the underlying topology to be lost.
  • Figure 3: (a) The relationship between data sampling ratio and relative data density. (b) The relationship between the sampling ratio and the k nearest neighbor preservation (k=15) of the resulting low-dimensional embedding.
  • Figure 4: Visualization Performance of ResNet Architecture.
  • Figure 5: Comparasion of visualization results on ResNet over FOOD101 dataset. The representations are shown with dots with color being their labels. The color of the background represents prediction and the white part is the decision boundary. For example, a blue sample lies in the red region indicates that a sample belonging to the blue class is misclassified by the model as red class. Note how SentryCam (b) maintains distinct, interpretable clusters for different classes, while DVI (a) collapses the data into a single pathological structure, obscuring class relationships.
  • ...and 10 more figures

Theorems & Definitions (4)

  • Theorem 1: Homotopy preservation under $\varepsilon$-density Hatcher200210.5555/3116660.3117013
  • Lemma 2: Equivalence up to constants
  • Corollary 3: Relative-density sufficient condition
  • Proposition 4: Phase transition cohen2005stabilitychazal2016structure