A Deep Ordinal Distortion Estimation Approach for Distortion Rectification

TL;DR

This work proposes a novel distortion rectification approach that can obtain more accurate parameters with higher efficiency, and is the first to unify the heterogeneous distortion parameters into a learning-friendly intermediate representation through ordinal distortion, bridging the gap between image feature and distort rectification.

Abstract

Distortion is widely existed in the images captured by popular wide-angle cameras and fisheye cameras. Despite the long history of distortion rectification, accurately estimating the distortion parameters from a single distorted image is still challenging. The main reason is these parameters are implicit to image features, influencing the networks to fully learn the distortion information. In this work, we propose a novel distortion rectification approach that can obtain more accurate parameters with higher efficiency. Our key insight is that distortion rectification can be cast as a problem of learning an ordinal distortion from a single distorted image. To solve this problem, we design a local-global associated estimation network that learns the ordinal distortion to approximate the realistic distortion distribution. In contrast to the implicit distortion parameters, the proposed ordinal distortion have more explicit relationship with image features, and thus significantly boosts the distortion perception of neural networks. Considering the redundancy of distortion information, our approach only uses a part of distorted image for the ordinal distortion estimation, showing promising applications in the efficient distortion rectification. To our knowledge, we first unify the heterogeneous distortion parameters into a learning-friendly intermediate representation through ordinal distortion, bridging the gap between image feature and distortion rectification. The experimental results demonstrate that our approach outperforms the state-of-the-art methods by a significant margin, with approximately 23% improvement on the quantitative evaluation while displaying the best performance on visual appearance. The code is available at https://github.com/KangLiao929/OrdinalDistortion.

Figures (13)

• Figure 1: Method Comparisons. (a) Previous learning methods, (b) Our proposed approach. We aim to transfer the traditional calibration objective into a learning-friendly representation. Previous methods roughly feed the whole distorted image into their learning models and directly estimate the implicit and heterogeneous distortion parameters. In contrast, our proposed approach only requires a part of a distorted image (distortion element) and estimates the ordinal distortion. Due to its explicit description and homogeneity, we can obtain more accurate distortion estimation and achieve better corrected results.
• Figure 2: Attributes of the proposed ordinal distortion. (a) Explicitness. The ordinal distortion is observable in an image and explicit to image features, which describes a series of distortion levels from small to large (top); the ordinal distortion always equals one in an undistorted image (bottom). (b) Homogeneity. Compared with the heterogeneous distortion parameters $\mathcal{K} = [k_1 \ \ k_2 \ \ k_3 \ \ k_4]$, the ordinal distortion $\mathcal{D} = [\delta_1 \ \ \delta_2 \ \ \delta_3 \ \ \delta_4]$ is homogeneous, representing the same concept of distortion distribution. (c) Redundancy. After different flip operations, although the semantic features of four patches have not any relevance (top), the ordinal distortion of four patches keeps the same in distribution with each other (bottom).
• Figure 3: Architecture of the proposed network. This network consists of a global perception module $M_{gp}$, local Siamese module $M_{ls}$, and distortion estimation module $M_{de}$. During the network's training process, we use four parts, i.e., distortion elements: $\Pi = [\pi_1 \ \ \pi_2 \ \ \pi_3 \ \ \pi_4]$ of the distorted image $I^d$ to train its ability of distortion perception sequentially, in which the distortion blocks: $\Theta = [\theta_1 \ \ \theta_2 \ \ \theta_3 \ \ \theta_4]$ derived from a distortion element provide the local distortion information. In the test or application stage, we only need one part of the input distorted image to estimate the ordinal distortion $\mathcal{D}$. Finally, the rectified image $I^r$ can be generated using the estimated ordinal distortion and the input distorted image.
• Figure 4: Motivation of the designed distortion-aware perception layer. Left: the distortion distribution map (DDM) that describes the degree of distortion for each pixel. Right: the corresponding distorted image. Particularly, we use the filters with increasing sizes to perceive the increasing degrees of distortions along the extended path (the white arrows).
• Figure 5: Comparison of two learning representations for distortion estimation, distortion parameter (left) and ordinal distortion (right). In contrast to the ambiguous relationship between the distortion distribution and distortion parameter, the proposed ordinal distortion displays an evident positive correlation to the distortion reprojection error.