Jointly-optimized Trajectory Generation and Camera Control for 3D Coverage Planning

TL;DR

This paper tackles 3D coverage planning with UAVs by jointly optimizing the vehicle's kinematics and the onboard camera controls to maximize the visible surface of a 3D object. It introduces a rolling-horizon finite-time optimal control approach that integrates ray-tracing for visibility, encoding the results as binary constraints within a mixed-integer program (MIQP) to compute look-ahead coverage trajectories. Key contributions include (i) a 3D coverage formulation on a triangular-mesh object, (ii) offline ray-tracing-based visibility constraints embedded into the optimization, and (iii) a memory mechanism to mitigate redundant coverage across successive horizons. The method is validated through extensive synthetic tests and a real-world 3D coverage mission, demonstrating improved coverage efficiency and the importance of visibility-aware planning for practical UAV missions.

Abstract

This work proposes a jointly optimized trajectory generation and camera control approach, enabling an autonomous agent, such as an unmanned aerial vehicle (UAV) operating in 3D environments, to plan and execute coverage trajectories that maximally cover the surface area of a 3D object of interest. Specifically, the UAV's kinematic and camera control inputs are jointly optimized over a rolling planning horizon to achieve complete 3D coverage of the object. The proposed controller incorporates ray-tracing into the planning process to simulate the propagation of light rays, thereby determining the visible parts of the object through the UAV's camera. This integration enables the generation of precise look-ahead coverage trajectories. The coverage planning problem is formulated as a rolling finite-horizon optimal control problem and solved using mixed-integer programming techniques. Extensive real-world and synthetic experiments validate the performance of the proposed approach.

Figures (14)

• Figure 1: The camera's FOV (with horizontal and vertical FOV angles given by $\varphi_h$ and $\varphi_v$ respectively) is represented by a regular right pyramid with height $h$, and a rectangular base of dimensions $(l, w)$. At each time-step $t$ a finite set of light-rays $\mathcal{L}(z_t,\theta_t,\phi_t,x_t^{\mathbf{p}}) = \{\Lambda_1,..,\Lambda_n\}$ enter the camera's FOV (shown with the dotted line segments) as discussed in Sec. \ref{['ssec:sensing_model']}
• Figure 2: The figure demonstrates various configurations of the camera's FOV at time $t$ with respect to the control inputs $z_t, \theta_t, \phi_t$ (i.e., zoom-level and rotation angles). In this figure the $z_t \in \{z_1,z_2\}$, $\theta_t \in \{\frac{\pi}{2}\}$, and $\phi_t \in \{0, \frac{\pi}{4},\frac{\pi}{2},\frac{3\pi}{4},\pi\}$ as shown.
• Figure 3: A 3D point-cloud representation $\mathcal{P} = \{p_1,..,p_{|\mathcal{P}|}\}$ of the object's surface $\partial \mathcal{W}$ is extracted, and then triangulated to form a triangle mesh $\mathcal{K}$ consisting of triangular facets $\kappa \in \mathcal{K}$ which need to be observed through the agent's camera.
• Figure 4: The figure shows that although the two points $p_1$ and $p_2$ (marked with $\ast$ and $\times$ respectively) reside inside the volume $\mathcal{V}_t$ covered by the agent's camera FOV, only point $p_1$ is visible. Specifically, in the camera configuration shown above there is no light-ray $\lambda \in \mathcal{L}(z_t,\theta_t,\phi_t,x_t^{\mathbf{p}})$ which traces back to point $p_2$ as all light-rays are blocked by the object of interest as shown.
• Figure 5: The figure illustrates the ray-tracing procedure discussed in Sec. \ref{['ssec:vis_con']} which is used in order to determine the visibility of some facet $\kappa \in \mathcal{K}$.