Set-membership target search and tracking within an unknown cluttered area using cooperating UAVs equipped with vision systems

TL;DR

This work develops a set-membership framework for cooperative search and tracking of an unknown number of moving ground targets in an unknown cluttered RoI using a fleet of CVS-equipped UAVs. It introduces CVS-derived hypotheses and a distributed estimation approach that yields guaranteed-set representations for identified targets and for yet-to-be-identified regions, while accounting for occlusions and unknown maps. Target locations and free-space are inferred from depth maps, pixel labels, and bounding boxes, and UAVs share information to refine estimates via a distributed MPC that minimizes overall uncertainty. Simulations in a simplified urban environment demonstrate promising target localization accuracy and substantial RoI coverage, while highlighting limitations due to occlusions and target dynamics, pointing to future work on richer 2.5D/3D mapping and decoy management.

Abstract

This paper addresses the problem of target search and tracking using a fleet of cooperating UAVs evolving in some unknown region of interest containing an a priori unknown number of moving ground targets. Each drone is equipped with an embedded Computer Vision System (CVS), providing an image with labeled pixels and a depth map of the observed part of its environment. Moreover, a box containing the corresponding pixels in the image frame is available when a UAV identifies a target. Hypotheses regarding information provided by the pixel classification, depth map construction, and target identification algorithms are proposed to allow its exploitation by set-membership approaches. A set-membership target location estimator is developed using the information provided by the CVS. Each UAV evaluates sets guaranteed to contain the location of the identified targets and a set possibly containing the locations of targets still to be identified. Then, each UAV uses these sets to search and track targets cooperatively.

Figures (15)

• Figure 1: Obstacle (in grey), target (in green), and $r^{\text{s}}$-ground neighborhood (in red) of its location.
• Figure 2: Left: Reference frame $\mathcal{F}$ and UAV body frame $\mathcal{F}_{i}^{\text{b}}$; Right: Camera frame $\mathcal{F}_{i}^{\text{c}}$ and body frame $\mathcal{F}_{i}^{\text{b}}$ of UAV $i$, when $\theta=0$
• Figure 3: Pinhole model of the camera and several light rays contributing to the illumination of the pixel $\left(n_{\text{r}},n_{\text{c}}\right)$; The $4$ blue lines are the light rays illuminating the corners of $\left(n_{\text{r}},n_{\text{c}}\right)$; The quadrangle $\mathbb{P}_{i}^{\text{g}}\left(\left(n_{\text{r}},n_{\text{c}}\right)\right)$ represents all points of the ground which may contribute to the illumination of pixel $\left(n_{\text{r}},n_{\text{c}}\right)$, see Section \ref{['sec:ExploitingCVS']}.
• Figure 4: Depth map information for one pixel: $\mathbf{D}_{i}^{0}\left(n_{\text{r}},n_{\text{c}}\right)$ is the distance between $\boldsymbol{x}_{i}^{\text{c}}$ and the red dot, while $\mathbf{D}_{i}\left(n_{\text{r}},n_{\text{c}}\right)$ represents the measured distance between $\boldsymbol{x}_{i}^{\text{c}}$ and the blue dot; The green interval is $\left[\mathbf{D}_{i}\right]\left(n_{\text{r}},n_{\text{c}}\right)$.
• Figure 5: As pixel $\boldsymbol{p}_{1}\in\mathcal{Y}_{i}^{\text{o}}$, all light rays illuminating $\boldsymbol{p}_{1}$ stem from an obstacle; as pixel $\boldsymbol{p}_{3}\in\mathcal{Y}_{i}^{\text{g}}$, all light rays illuminating $\boldsymbol{p}_{3}$ stem from $\mathbb{X}_{\text{g}}$; As $\boldsymbol{p}_{2}\in\mathcal{Y}_{i}^{\text{n}}$, nothing can be concluded.

Theorems & Definitions (7)

• Proposition 1
• Corollary 2
• Proposition 3
• Proposition 4
• Proposition 5
• Proposition 6
• Proposition 7