Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model Proficiency in Centralized Multi-Agent Systems: A Performance Study

Anna Guerra, Francesco Guidi, Pau Closas, Davide Dardari, Petar M. Djuric

TL;DR

The paper extends proficiency self-assessment (PSA) to centralized multi-agent systems by introducing three metrics—Measurement Prediction Bound (MPB), Kolmogorov-Smirnov (KS) statistic, and Kullback-Leibler (KL) divergence—to quantify discrepancies between predicted and actual measurements during operation. MPB provides a Bayesian information-theoretic bound on prediction error, KS enables hypothesis-testing via p-values on predictive-distribution differences, and KL serves as a theoretical benchmark for model mismatch. Through a nonlinear target-tracking case study with an EKF, the authors demonstrate that MPB and KS reliably detect model mismatches and align with KL divergence, enabling real-time proficiency assessment without ground-truth state. The work offers practical, computation-friendly tools for team-level PSA in centralized networks and lays groundwork for extending to distributed multi-agent configurations.

Abstract

Autonomous agents are increasingly deployed in dynamic environments where their ability to perform a given task depends on both individual and team-level proficiency. While proficiency self-assessment (PSA) has been studied for single agents, its extension to a team of agents remains underexplored. This letter addresses this gap by presenting a framework for team PSA in centralized settings. We investigate three metrics for centralized team PSA: the measurement prediction bound (MPB), the Kolmogorov-Smirnov (KS) statistic, and the Kullback-Leibler (KL) divergence. These metrics quantify the discrepancy between predicted and actual measurements. We use the KL divergence as a reference metric since it compares the true and predictive distributions, whereas the MPB and KS provide efficient indicators for in situ assessment. Simulation results in a target tracking scenario demonstrate that both MPB and KS metrics accurately capture model mismatches, align with the KL divergence reference, and enable real-time proficiency assessment.

Model Proficiency in Centralized Multi-Agent Systems: A Performance Study

TL;DR

The paper extends proficiency self-assessment (PSA) to centralized multi-agent systems by introducing three metrics—Measurement Prediction Bound (MPB), Kolmogorov-Smirnov (KS) statistic, and Kullback-Leibler (KL) divergence—to quantify discrepancies between predicted and actual measurements during operation. MPB provides a Bayesian information-theoretic bound on prediction error, KS enables hypothesis-testing via p-values on predictive-distribution differences, and KL serves as a theoretical benchmark for model mismatch. Through a nonlinear target-tracking case study with an EKF, the authors demonstrate that MPB and KS reliably detect model mismatches and align with KL divergence, enabling real-time proficiency assessment without ground-truth state. The work offers practical, computation-friendly tools for team-level PSA in centralized networks and lays groundwork for extending to distributed multi-agent configurations.

Abstract

Autonomous agents are increasingly deployed in dynamic environments where their ability to perform a given task depends on both individual and team-level proficiency. While proficiency self-assessment (PSA) has been studied for single agents, its extension to a team of agents remains underexplored. This letter addresses this gap by presenting a framework for team PSA in centralized settings. We investigate three metrics for centralized team PSA: the measurement prediction bound (MPB), the Kolmogorov-Smirnov (KS) statistic, and the Kullback-Leibler (KL) divergence. These metrics quantify the discrepancy between predicted and actual measurements. We use the KL divergence as a reference metric since it compares the true and predictive distributions, whereas the MPB and KS provide efficient indicators for in situ assessment. Simulation results in a target tracking scenario demonstrate that both MPB and KS metrics accurately capture model mismatches, align with the KL divergence reference, and enable real-time proficiency assessment.

Paper Structure

This paper contains 9 sections, 16 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Research Problem
  3. Proficiency Analysis
  4. Measurement Prediction Bound (MPB)
  5. The Kolmogorov-Smirnov (KS) Statistics
  6. The Kullback-Leibler Divergence
  7. Case Study: Proficiency Evaluation for Target Tracking
  8. Simulation Results
  9. Conclusions

Figures (2)

  • Figure 1: Inverse of the team MPB according to the selected models and as a function of time. Markers plot the error ${e}_t^2\left(\mathcal{M}\right)$ obtained from a single simulation trial, and dotted lines depict the MSE averaged over $100$ trials.
  • Figure 2: Left: $\bar{d}_{\mathrm{MPB}, t}$ as a function of $t$ and $\mathcal{M}$. Middle: Sequence of averaged $p$-values for the KS. Right: KL divergence. Continuous lines represent the distance averaged over $N_{\text{MC}}=100$, whereas markers refer to a single simulation trial.