PLATE: A perception-latency aware estimator,

TL;DR

PLATE tackles real-time target tracking under a perception-latency/accuracy trade-off by leveraging a bank of perception methods with different latencies. It formulates a latency-aware estimation-and-scheduling problem, solves it exactly via dynamic programming and efficiently via a quantized-covariance approximation, and extends to online moving-horizon operation. The approach yields provable guarantees on the sparsity of the search space and near-optimal schedules, while enabling substantial reductions in CPU load and perception attention without sacrificing accuracy. Empirical validation on simulations and MOT16 data shows PLATE outperforms static and heuristic baselines, highlighting its practical impact for resource-constrained, latency-sensitive tracking systems.

Abstract

Target tracking is a popular problem with many potential applications. There has been a lot of effort on improving the quality of the detection of targets using cameras through different techniques. In general, with higher computational effort applied, i.e., a longer perception-latency, a better detection accuracy is obtained. However, it is not always useful to apply the longest perception-latency allowed, particularly when the environment doesn't require to and when the computational resources are shared between other tasks. In this work, we propose a new Perception-LATency aware Estimator (PLATE), which uses different perception configurations in different moments of time in order to optimize a certain performance measure. This measure takes into account a perception-latency and accuracy trade-off aiming for a good compromise between quality and resource usage. Compared to other heuristic frame-skipping techniques, PLATE comes with a formal complexity and optimality analysis. The advantages of PLATE are verified by several experiments including an evaluation over a standard benchmark with real data and using state of the art deep learning object detection methods for the perception stage.

Figures (7)

• Figure 1: Graphical depiction of Algorithm \ref{['algo:dyn_prog']}. At each step, all scheduling method options are explored in a recursive fashion with a maximum depth until the condition $\sum_{k=0}^{\alpha}\Delta{p_k}\geq T_f$ where $\alpha={\normalfont\textsf{att}}(p;[0,T_f])$ is reached.
• Figure 2: Transition graph example for covariance evolution using $\mathcal{G}$. In this example $D=2$. Each node labeled in $1,\dots,Q(\delta)$ is connected to other nodes through one of the two perception decisions, either one or two steps ahead since $\Delta^1=\Delta_s, \Delta^2=2\Delta_s$ for this example.
• Figure 3: Covariance norm evolution, showing the bound $B_s$ as well as $\|\hat{\mathbf{P}}(\tau_k)\|$ for PLATE with 100 random initial conditions $\hat{\mathbf{P}}[0]$ and schedules $p$ with $\|\hat{\mathbf{P}}[0]\|_F\leq B_0=1$.
• Figure 4: Resulting histograms for the cost difference $|J_{\min}-J_p|$ with the schedule obtained from the approximate dynamic programming approach for $100$ random initial conditions for $P[0]$ as described in Section \ref{['sec:histogram']}. The parameters $T, \lambda_\alpha$ of the cost function in \ref{['eq:cost']} were changed for three different scenarios as well as the number of quantization points $Q(\delta)$. In addition, only the average CPU load for all experiments is shown for convenience.
• Figure 5: Behaviour of the target model \ref{['eq:system_sde']} and the output of PLATE using the moving-horizon scheduling with $T_f=10$ and $Q(\delta)=5000$ when an occlusion occurs for $t\in[4,6]$. Moreover, $I_x(t)$ and $I_{v_x}(t)$ represent confidence intervals around $\hat{x}(t)$ and $\hat{v}_x(t)$ of 3 times their standard deviation.

Theorems & Definitions (14)

• Proposition 1
• Theorem 1
• Proposition 2
• Proposition 3
• Proposition 4
• Theorem 2
• Proposition 5
• Lemma 1
• Lemma 2
• Lemma 3