See Behind Walls in Real-time Using Aerial Drones and Augmented Reality

TL;DR

ARD^2 presents a real-time through-wall surveillance framework using two aerial drones and an AR headset. It decomposes the problem into target direction estimation via geometric Relationships and contour reconstruction through few-shot view synthesis augmented with simulated data and on-site calibration. The system achieves a direction error of $2.69^\circ$ and contour reconstruction error as low as $5.96\%$, with end-to-end latency suggesting real-time AR overlays on capable hardware. This approach offers a promising see-through-obstacle capability for security, law enforcement, and tactical operations, while acknowledging two-view limitations and the need for target priors or additional drones for broader applicability.

Abstract

This work presents ARD2, a framework that enables real-time through-wall surveillance using two aerial drones and an augmented reality (AR) device. ARD2 consists of two main steps: target direction estimation and contour reconstruction. In the first stage, ARD2 leverages geometric relationships between the drones, the user, and the target to project the target's direction onto the user's AR display. In the second stage, images from the drones are synthesized to reconstruct the target's contour, allowing the user to visualize the target behind walls. Experimental results demonstrate the system's accuracy in both direction estimation and contour reconstruction.

Figures (10)

• Figure 1: Application scenario
• Figure 2: Two-step workflow
• Figure 3: Extreme case illustration
• Figure 4: Input contour scaling
• Figure 5: Raw image, input, output, and final results