Success Probability in Multi-View Imaging

TL;DR

This work addresses the probabilistic nature of achieving reliable 3D reconstruction in multi-view imaging when platform pointing is noisy. It introduces a graph-based framework that connects field-of-view overlap, visual similarity, and self-calibration feasibility to a Monte Carlo estimate of the success probability, $P_{calib}$, for a given number of views and pointing error $APE$. The approach defines overlap metrics $AO$ and $RO$, and uses connected components to quantify the likelihood that a subset of views can be geometrically self-calibrated, enabling tomography or CT of the observed domain. Practical implications are demonstrated with a nanosatellite cloud tomography scenario (CloudCT), revealing strong tradeoffs between $APE$, FOV footprint, and the number of viewpoints needed to achieve a desired success rate, and suggesting design choices to balance efficiency and resolution in formation-flying missions.

Abstract

Platforms such as robots, security cameras, drones and satellites are used in multi-view imaging for three-dimensional (3D) recovery by stereoscopy or tomography. Each camera in the setup has a field of view (FOV). Multi-view analysis requires overlap of the FOVs of all cameras, or a significant subset of them. However, the success of such methods is not guaranteed, because the FOVs may not sufficiently overlap. The reason is that pointing of a camera from a mount or platform has some randomness (noise), due to imprecise platform control, typical to mechanical systems, and particularly moving systems such as satellites. So, success is probabilistic. This paper creates a framework to analyze this aspect. This is critical for setting limitations on the capabilities of imaging systems, such as resolution (pixel footprint), FOV, the size of domains that can be captured, and efficiency. The framework uses the fact that imprecise pointing can be mitigated by self-calibration - provided that there is sufficient overlap between pairs of views and sufficient visual similarity of views. We show an example considering the design of a formation of nanosatellites that seek 3D reconstruction of clouds.

Figures (9)

• Figure 1: Pointing errors across multi-view platforms: [Left] Satellites having alignment challenges during Earth observation. [Middle] Drones having orientation noise. [Right] The same phenomenon can be seen in ground-based (e.g, security) cameras.
• Figure 2: A general multi-view setup. One of the cameras is the anchor. Its axes and location correspond to the setup coordinate system. Each camera has a FOV on the domain surface.
• Figure 3: [Left] Disconnected views (nodes). In this example, there are two graph components, ${\rm B}({\rm Q}=3)$ and ${\rm B}({\rm Q}=2)$. [Right] A graph of a connected set of views. Connection is marked by an edge. An edge between nodes $c$ and $c'$ means that ${\rm RO}_{c, c'}>{\rm T}$ and $\mu_{c, c'} \le \mu_{\rm max}$. The graph has a single component ${\rm B}({\rm Q}=5)$.
• Figure 4: Example of estimation of components of ${\rm P}_{\rm calib}$ via Monte Carlo with $N_{\rm MC}=5$.
• Figure 5: [Left] The camera coordinate system and the effect of noisy pointing. [Middle] Four rays bound the FOV frustum in 3D. [Right] The camera FOV frustum, image sensor plane, normalized image plane and angular FOV parameters.