Efficient On-Board Processing of Oblique UAV Video for Rapid Flood Extent Mapping

TL;DR

The paper tackles the challenge of real-time, on-board semantic segmentation of high-resolution oblique UAV video under SWaP constraints. It introduces Temporal Token Reuse (TTR), a framework that treats image patches as tokens and caches deep features for temporally static regions to avoid recomputing them, coupled with SegBlocks for patch-based adaptive CNN processing. The approach leverages a lightweight cosine-similarity-based change detector and a per-layer feature cache, enabling significant speedups (up to ~30% latency reduction) with negligible accuracy loss on edge hardware across multiple UAV benchmarks, including a newly curated Floodwater dataset designed for real-time flood mapping. This work shifts the accuracy–efficiency Pareto frontier for onboard flood extent mapping and demonstrates practical applicability for time-critical remote sensing missions.

Abstract

Effective disaster response relies on rapid disaster response, where oblique aerial video is the primary modality for initial scouting due to its ability to maximize spatial coverage and situational awareness in limited flight time. However, the on-board processing of high-resolution oblique streams is severely bottlenecked by the strict Size, Weight, and Power (SWaP) constraints of Unmanned Aerial Vehicles (UAVs). The computational density required to process these wide-field-of-view streams precludes low-latency inference on standard edge hardware. To address this, we propose Temporal Token Reuse (TTR), an adaptive inference framework capable of accelerating video segmentation on embedded devices. TTR exploits the intrinsic spatiotemporal redundancy of aerial video by formulating image patches as tokens; it utilizes a lightweight similarity metric to dynamically identify static regions and propagate their precomputed deep features, thereby bypassing redundant backbone computations. We validate the framework on standard benchmarks and a newly curated Oblique Floodwater Dataset designed for hydrological monitoring. Experimental results on edge-grade hardware demonstrate that TTR achieves a 30% reduction in inference latency with negligible degradation in segmentation accuracy (< 0.5% mIoU). These findings confirm that TTR effectively shifts the operational Pareto frontier, enabling high-fidelity, real-time oblique video understanding for time-critical remote sensing missions

Figures (9)

• Figure 1: We visualize four pairs of patches to demonstrate high self-similarity over time. For instance, the sky patch in (a) and the water patch in (b) remain almost identical between Frame 10 and Frame 250. Similarly, patches for water (c) and vegetation (d) show negligible change over hundreds of frames despite camera motion. Our TTR framework leverages a high cosine similarity score between such patches to bypass their re-computation.
• Figure 2: A detailed operational diagram of the TTR framework. The data flow bifurcates based on patch similarity: (1) Changed patches undergo full CNN processing, with the new features used to update the cache. (2) For redundant patches, features are retrieved directly from the cache, bypassing the CNN. The final mask is assembled from both new and reused features.
• Figure 3: The trade-off between segmentation accuracy (mIoU, blue) and computational savings (% Blocks Reused, red). As the threshold becomes stricter (moving right), accuracy improves at the cost of reduced savings. We selected $\tau=0.99$ as our operating point, offering a strong balance between high performance and significant computational reduction.
• Figure 4: For each scene, the top row shows the original aerial RGB image. The middle row provides the corresponding pixel-level ground truth annotation provided by the dataset. The bottom row displays the segmentation masks laid on top of the RGB images.
• Figure 5: Visualizations of A2D2 data. From left: semantic segmentation, 3D bounding boxes, dense point cloud from SLAM, single frame point cloud overlaid on corresponding camera image