Visibility in Polygonal Environments with Holes: Finding Best Spots for Hiding and Surveillance

TL;DR

The paper addresses hiding and surveillance in polygonal environments by minimizing a non-smooth visibility metric $V(x)=\|S(x)\cap D_2\|_\mu$, where $S(x)$ is the visibility polygon from $x$. It develops a rigorous non-smooth analytic framework, introducing anchors and inflection segments to characterize critical points, and extends regularity results to limited range and field of view. To solve the resulting non-convex, non-differentiable optimization problem, the Normalized Descent (Norcent) algorithm is proposed, leveraging μ-local Lipschitz properties and randomization to avoid saddle points, with provable almost-sure convergence to local minima. Simulations in hide-and-seek scenarios validate the approach and demonstrate practical effectiveness in identifying low-detection regions and effective surveillance positions. The work advances interpretable, geometry-aware tools for visibility-based decision making in uncertain, obstacle-rich environments, with potential applications in autonomous robotics and multi-agent coordination.

Abstract

Visibility plays an important role for decision making in cluttered, uncertain environments. This paper considers the problem of identifying optimal hiding spots for an agent against line-of-sight detection by an adversary whose location is unknown. We consider environments modeled as polygons with holes. We develop a set of mathematical tools for reasoning about visibility as a function of position and rely on non-smooth analysis to formally characterize the regularity properties of various visibility-based metrics. These metrics are non-smooth and non-convex, so off-the-shelf optimization algorithms can only guarantee convergence to Clarke critical points. To address this, the proposed Normalized Descent algorithm leverages the structure of non-smooth points in visibility problems and introduces randomness to escape saddle points. Our technical analysis allows for the non-monotonic decrease in the visibility metric and strengthens the algorithm guarantees, ensuring convergence to local minima with high probability. Simulations on two hide-and-seek scenarios showcase the effectiveness of the proposed approach.

Figures (12)

• Figure 1: (a) For an observer at $x$, its visibility region $S(x)$ is shaded in blue. For the free space $\mathcal{F}$, we have reflex vertices $\mathrm{Ve}_r(\mathcal{F}) = \{ q^3, q^6, o^1, o^2, o^3, o^4\}$. The reflex angles are shown in violet and the non-reflex angles are shown in orange. (b) Visualization of notions on projected rays and associated distances. The ray cast from $x$ are shown in thin lines, while the projected rays are shown in bold ones.
• Figure 1: In this example, with $D_1$ shaded in green, and $D_2$ shaded in orange, $x_1$ is a local minimizer as $0 = V(x_1) \leqslant V(y), \forall y \in \mathbb{B}_{\epsilon}(x_1) \operatorname{\cap} D_1$. Analogously, $x_2$ is a local maximizer as $V(x_2) \geqslant V(y), \forall y \in \mathbb{B}_{\epsilon}(x_2) \operatorname{\cap} D_1$.
• Figure 1: An observer at $x$ has anchors $\mathrm{Ve}_a(x) = \left\{ o^1, o^2, o^4 \right\}$. Anchors $o^1, o^4$ are positively oriented (showed in red) and $o^2$ is negatively oriented wrt $x$ (shown in green).
• Figure 1: (a) Inflection curves for visibility for different ranges. The range of the observer is $R = 1.2$ to the left of the dotted line and $R = 2.5$ to the right. (b) Inflection curves for visibility with different ranges and fields of view. The observer is facing west to the left of the dotted line, with range $R=1.5$ and field of view angle $\varphi=120^\circ$. The observer is facing south-east to the right of the dotted line, with range $R=2.5$ and field of view angle $\varphi=30^\circ$.
• Figure 1: Hiding from an quadrotor. The climbers can freely move along the terrain (green), while the quadrotor is restricted to a certain height range (red). (b) also shows the trajectories of the Norcent algorithm for two different initial positions of the climber.

Theorems & Definitions (55)

• Lemma 2.1
• Proof 1
• Lemma 2.2: Intersection of μ-locally Lipschitz set-valued maps
• Proof 2
• Lemma 2.3: Local extrema
• Lemma 4.1: Construction of the visibility polygon
• Lemma 4.2: When is a reflex vertex an anchor?
• Proof 3
• Definition 4.3: Inflection Segments
• Lemma 4.4: Bound on the number of inflection segments