Towards Causal Physical Error Discovery in Video Analytics Systems

TL;DR

This paper tackles debugging video analytics systems by moving from intrinsic pixel explanations to extrinsic causal explanations that reflect real-world phenomena. It introduces VizEx, a framework that uses regression discontinuity design ($RDD$) to identify causal links between low-dimensional KPIs, derived from video and external streams, and model errors. The authors outline the system architecture, KPI definitions ($K_i(\lambda,w)$), a query mechanism with a BECAUSE clause, and preliminary VIRAT-based experiments, arguing that surrogate models are insufficient for robust causal debugging. The work lays groundwork for actionable, scalable debugging that can guide data collection and preprocessing in fixed-camera video deployments.

Abstract

Video analytics systems based on deep learning models are often opaque and brittle and require explanation systems to help users debug. Current model explanation system are very good at giving literal explanations of behavior in terms of pixel contributions but cannot integrate information about the physical or systems processes that might influence a prediction. This paper introduces the idea that a simple form of causal reasoning, called a regression discontinuity design, can be used to associate changes in multiple key performance indicators to physical real world phenomena to give users a more actionable set of video analytics explanations. We overview the system architecture and describe a vision of the impact that such a system might have.

Figures (3)

• Figure 1: Saliency maps are a common explanation tool for computer vision models, which provide pixel-level contributions to a final output.
• Figure 2: YOLOv3 undercounts and overcounts "people" in very specific areas of a scene corresponding to occlusion, disorder, or lighting.
• Figure 3: Average luminosity value & accuracy for each frame