Investigation of moving objects through atmospheric turbulence from a non-stationary platform

Abstract

In this work, we extract the optical flow field corresponding to moving objects from an image sequence of a scene impacted by atmospheric turbulence \emph{and} captured from a moving camera. Our procedure first computes the optical flow field and creates a motion model to compensate for the flow field induced by camera motion. After subtracting the motion model from the optical flow, we proceed with our previous work, Gilles et al~\cite{gilles2018detection}, where a spatial-temporal cartoon+texture inspired decomposition is performed on the motion-compensated flow field in order to separate flows corresponding to atmospheric turbulence and object motion. Finally, the geometric component is processed with the detection and tracking method and is compared against a ground truth. All of the sequences and code used in this work are open source and are available by contacting the authors.

Figures (15)

• Figure 1: Frame 30 from the Courtyard4 sequence is shown (left) along with its corresponding version with the introduction of simulated atmospheric turbulence (center). On the right is a close view of the remote control car used in each sequence.
• Figure 2: Illustration of the concept of optical flow. The left image represents an object (the red area) as its initial position centered around $(x_0,y_0)$ and its position in the next frame (the dashed area) centered around $(x_1,y_1)$. In the center figure, the set of arrows corresponds to the motion vectors i.e. optical flow. The colorwheel on the right provides the correspondence between color and movement direction.
• Figure 3: Representations of the optical flow field corresponding to the Courtyard4 sequence at frame 30 without the inclusion of simulated turbulence.
• Figure 4: Demonstration of empirical motion model results when applied to Courtyard4 at frame 30 with the inclusion of atmospheric turbulence. The $x$ and $y$ component are shown on the top and bottom left respectively. In the middle is the corresponding components of the camera motion model. The compensated optical flow is shown for each component to the right.
• Figure 5: Demonstration of camera motion subtraction method from Courtyard4 at frame 30 with simulated atmospheric turbulence. On the left, we see the image corresponding to frame 30 which has observable deformation due to simulated atmospheric turbulence. To its right, we see the corresponding optical flow where the impact of atmospheric turbulence is seen with the majority of the flow field occluded by camera motion. Next, we have the camera motion model that is computed by the empirical method. Finally, we have the camera motion-compensated optical flow field on the right, where we see only the impact of atmospheric turbulence.