Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CSAOT: Cooperative Multi-Agent System for Active Object Tracking

Hy Nguyen, Bao Pham, Hung Du, Srikanth Thudumu, Rajesh Vasa, Kon Mouzakis

TL;DR

CSAOT tackles Active Object Tracking by introducing a cooperative multi-agent reinforcement learning framework that runs on a single device. It decomposes the task into three role-based agents—Detection, Obstacle, and Movement—whose outputs are combined by a Mixture of Policies (MoP) to yield the final navigation action, aided by subtask-based rewards and PPO optimization. The two-layer architecture, with ResNet50-based perception and LSTM memory, enables efficient, real-time decision-making and improved handling of occlusions and rapid target motion. Experiments in the AirSim simulator across four maps show CSAOT outperforms a single-agent baseline in complex scenarios, while reducing hardware costs and demonstrating potential for real-world AOT applications.

Abstract

Object Tracking is essential for many computer vision applications, such as autonomous navigation, surveillance, and robotics. Unlike Passive Object Tracking (POT), which relies on static camera viewpoints to detect and track objects across consecutive frames, Active Object Tracking (AOT) requires a controller agent to actively adjust its viewpoint to maintain visual contact with a moving target in complex environments. Existing AOT solutions are predominantly single-agent-based, which struggle in dynamic and complex scenarios due to limited information gathering and processing capabilities, often resulting in suboptimal decision-making. Alleviating these limitations necessitates the development of a multi-agent system where different agents perform distinct roles and collaborate to enhance learning and robustness in dynamic and complex environments. Although some multi-agent approaches exist for AOT, they typically rely on external auxiliary agents, which require additional devices, making them costly. In contrast, we introduce the Collaborative System for Active Object Tracking (CSAOT), a method that leverages multi-agent deep reinforcement learning (MADRL) and a Mixture of Experts (MoE) framework to enable multiple agents to operate on a single device, thereby improving tracking performance and reducing costs. Our approach enhances robustness against occlusions and rapid motion while optimizing camera movements to extend tracking duration. We validated the effectiveness of CSAOT on various interactive maps with dynamic and stationary obstacles.

CSAOT: Cooperative Multi-Agent System for Active Object Tracking

TL;DR

CSAOT tackles Active Object Tracking by introducing a cooperative multi-agent reinforcement learning framework that runs on a single device. It decomposes the task into three role-based agents—Detection, Obstacle, and Movement—whose outputs are combined by a Mixture of Policies (MoP) to yield the final navigation action, aided by subtask-based rewards and PPO optimization. The two-layer architecture, with ResNet50-based perception and LSTM memory, enables efficient, real-time decision-making and improved handling of occlusions and rapid target motion. Experiments in the AirSim simulator across four maps show CSAOT outperforms a single-agent baseline in complex scenarios, while reducing hardware costs and demonstrating potential for real-world AOT applications.

Abstract

Object Tracking is essential for many computer vision applications, such as autonomous navigation, surveillance, and robotics. Unlike Passive Object Tracking (POT), which relies on static camera viewpoints to detect and track objects across consecutive frames, Active Object Tracking (AOT) requires a controller agent to actively adjust its viewpoint to maintain visual contact with a moving target in complex environments. Existing AOT solutions are predominantly single-agent-based, which struggle in dynamic and complex scenarios due to limited information gathering and processing capabilities, often resulting in suboptimal decision-making. Alleviating these limitations necessitates the development of a multi-agent system where different agents perform distinct roles and collaborate to enhance learning and robustness in dynamic and complex environments. Although some multi-agent approaches exist for AOT, they typically rely on external auxiliary agents, which require additional devices, making them costly. In contrast, we introduce the Collaborative System for Active Object Tracking (CSAOT), a method that leverages multi-agent deep reinforcement learning (MADRL) and a Mixture of Experts (MoE) framework to enable multiple agents to operate on a single device, thereby improving tracking performance and reducing costs. Our approach enhances robustness against occlusions and rapid motion while optimizing camera movements to extend tracking duration. We validated the effectiveness of CSAOT on various interactive maps with dynamic and stationary obstacles.
Paper Structure (27 sections, 9 equations, 5 figures, 4 tables)

This paper contains 27 sections, 9 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Object tracking task and existing solutions
  4. Multi-agent Deep Reinforcement Learning
  5. Mixture of Experts
  6. Preliminaries
  7. Problem formulation
  8. Task formulation
  9. Mixture of Experts
  10. Proximal Policy Optimization
  11. CSAOT Framework
  12. Overall of the framework
  13. Framework architecture
  14. Reward construction
  15. Novel adaptations
  16. ...and 12 more sections

Figures (5)

  • Figure 1: CSAOT framework: Each agent (Detection Agent, Obstacle Agent, Movement Agent) is responsible for different tasks and operates on the same device, using observations from the same source. The final navigation agent takes the outputs of these three agents, along with the original observation, as its input and produces the navigation actions for the tracker. Each agent is implemented using a Mixture of Policy (MoP) mechanism.
  • Figure 2: Environment visualization, in clear weather mode
  • Figure 3: Target path for testing (White line), from left to right: Complex, SharpLoop, SimpleLoop, SingleTurn
  • Figure 4: Sample decision in Complex map. Steer and acceleration values are 0.398 and 0.44, respectively
  • Figure 5: EL during training with CSAOT in Complex map