Deployment Prior Injection for Run-time Calibratable Object Detection

TL;DR

This work introduces an additional graph input to the detector, where the graph represents the deployment context prior, and its edge values represent object relations, hence calibrating, or "re-biasing" the detector towards the given prior at run-time without parameter update.

Abstract

With a strong alignment between the training and test distributions, object relation as a context prior facilitates object detection. Yet, it turns into a harmful but inevitable training set bias upon test distributions that shift differently across space and time. Nevertheless, the existing detectors cannot incorporate deployment context prior during the test phase without parameter update. Such kind of capability requires the model to explicitly learn disentangled representations with respect to context prior. To achieve this, we introduce an additional graph input to the detector, where the graph represents the deployment context prior, and its edge values represent object relations. Then, the detector behavior is trained to bound to the graph with a modified training objective. As a result, during the test phase, any suitable deployment context prior can be injected into the detector via graph edits, hence calibrating, or "re-biasing" the detector towards the given prior at run-time without parameter update. Even if the deployment prior is unknown, the detector can self-calibrate using deployment prior approximated using its own predictions. Comprehensive experimental results on the COCO dataset, as well as cross-dataset testing on the Objects365 dataset, demonstrate the effectiveness of the run-time calibratable detector.

Figures (32)

• Figure 1: Calibratable Object Detector that allows Deployment Prior Injection at Run-time. The detector exposes a graph structure where the nodes are objects, while the edges are object relations. The model behavior is consistent with the graph structure, and hence deployment priors can be injected as graph edits.
• Figure 2: A detector detr implicitly learns object relations via entangled representations. Since $P(\text{person }|\text{ baseball glove})=99.0\%$ in the COCO training set, the gradient norm (shown on the right side) of the "baseball glove" show the shape of "person". The gradient norm is visualized following the attribution method in deformabledetr.
• Figure 3: The architecture of our proposed run-time calibratable object detector (abbr., CaliDet). Injection of deployment prior is achieved by editing the edges $E$ of the graph for the model. It can be used in DETR-like detectors deformabledetrdetrdino, such as DINO dino. This figure shows the CaliFormer for calibration vectors $V'$ which is elaborated in Section \ref{['sec:31']}. The other components in our model are elaborated in Section \ref{['sec:32']}.
• Figure 4: Demonstration of Logit Manipulation Loss (LoMa).
• Figure 5: Visualization of $E_{t}$, $E_{v}$, and $E_{v}-E_{t}$. The mean absolute error between the two matrices is $\epsilon=\sum\text{abs}(E_{v}-E_{t})/K^{2}=0.008$. The 0-th, 50-th, 90-th, 97-th, 100-th percentile values of $\text{abs}(E_{v}-E_{t})$ are $0.0,$$0.003,$$0.021$, $0.044$, $0.272$, respectively.