Affine Correspondences in Stereo Vision: Theory, Practice, and Limitations

TL;DR

The fundamental statements for affine transformations and epipolar geometry are overviewed and it is concluded that the estimation accuracy is around a few degrees for realistic test cases.

Abstract

Affine transformations have been recently used for stereo vision. They can be exploited in various computer vision application, e.g., when estimating surface normals, homographies, fundamental and essential matrices. Even full 3D reconstruction can be obtained by using affine correspondences. First, this paper overviews the fundamental statements for affine transformations and epipolar geometry. Then it is investigated how the transformation accuracy influences the quality of the 3D reconstruction. Besides, we propose novel techniques for estimating the local affine transformation from corresponding image directions; moreover, the fundamental matrix, related to the processed image pair, can also be exploited. Both synthetic and real quantitative evaluations are implemented based on the accuracy of the reconstructed surface normals. For the latter one, a special object, containing three perpendicular planes with chessboard patterns, is constructed. The quantitative evaluations are based on the accuracy of the reconstructed surface normals and it is concluded that the estimation accuracy is around a few degrees for realistic test cases. Special stereo poses and plane orientations are also evaluated in detail.

Figures (10)

• Figure 1: Two images of a spatial scene. A small planar surface is perspectively projected to images $P_1$ and $P_2$. The shape deformation between projected patches are transformed by affine transformation $\mathbf A$. Green: Point cloud can be reconstructed from PCs. Green+Red: Oriented point cloud obtained from ACs
• Figure 2: An affine transformation transforms the shape of the red quadrilaterals from the first to the second images. It also connects normals of corresponding epipolar lines. The direction and scale changes of normals $\mathbf n_1$ and $\mathbf n_2$ are determined by the affine transformation. $\mathbf C_1$, $\mathbf C_2$, $\mathbf e_1$, and $\mathbf e_2$ are focal points and epipoles, respectively.
• Figure 3: Block diagram of proposed pipeline for AC-based 3D reconstruction. This pipeline is designed for the quantitative evaluation. Its input are only the chessboard corners. The output are the reconstructed oriented point clouds: spatial locations and normals of reconstructed surfaces.
• Figure 4: Left. Three different directions are considered, each connects chessboard corners: horizontal (red), vertical (green), diagonal (purple). The horizontal and vertical directions are exploited for affine transformation estimation when two directions are used (methods F2UDIR and 2SDIR); diagonal directions are applied for three direction-based estimation (methods F3UDIR, DET3UDIR, 3SDIR). For the diagonal direction, only one out of the two possible directions is utilized. Right. The structure of the planes for special motions. Plane #1, #2, #3 are perpendicular to the axes $Z$, $Y$, and $X$, respectively.
• Figure 5: Directional error of the normals w.r.t. plane normals. General stereo pose is considered. Directional noise, given in degrees, is added to the angle of 2D directions. The affine transformations are computed by the F2UDIR algorithm. The average error for all planes are plotted.