RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation

TL;DR

A distortion vector map (DVM) is proposed that measures the degree and direction of local distortion and promotes the model performance in rectifying both central and deviated fisheye images, compared with models trained on single-type fisheye images.

Abstract

Fisheye images are categorized fisheye into central and deviated based on the optical center position. Existing rectification methods are limited to central fisheye images, while this paper proposes a novel method that extends to deviated fisheye image rectification. The challenge lies in the variant global distortion distribution pattern caused by the random optical center position. To address this challenge, we propose a distortion vector map (DVM) that measures the degree and direction of local distortion. By learning the DVM, the model can independently identify local distortions at each pixel without relying on global distortion patterns. The model adopts a pre-training and fine-tuning training paradigm. In the pre-training stage, it predicts the distortion vector map and perceives the local distortion features of each pixel. In the fine-tuning stage, it predicts a pixel-wise flow map for deviated fisheye image rectification. We also propose a data augmentation method mixing central, deviated, and distorted-free images. Such data augmentation promotes the model performance in rectifying both central and deviated fisheye images, compared with models trained on single-type fisheye images. Extensive experiments demonstrate the effectiveness and superiority of the proposed method.

Figures (8)

• Figure 1: Examples of a central fisheye image and a deviated fisheye image, together with their rectified images.
• Figure 2: Distortion patterns of two fisheye images are illustrated by their Distortion Vector Maps.
• Figure 3: Framework of our method for fisheye image rectification. It consists of two stages: (I) supervised pre-training that learns the local distortion features of fisheye images through a DVM estimation pretext task. (II) fine-tuning for rectification which leverages the learned representation to reconstruct the rectified image with a pixel-wise flow map.
• Figure 4: The internal structure of the DaFIR encoder and the DaFIR decoder.
• Figure 5: Qualitative comparison on the synthesized deviated fisheye images. From left to right, the sequence is as follows: the distorted image, DaFIR liao-et-al:DaFIR, PCN yang-et-al:progressively, DR-GAN liao-et-al:dr, ModelFree liao-et-al:model, MLC liao-et-al:multi, SimFIR feng-et-al:simfir, our method, and the ground truth.