Deep-BrownConrady: Prediction of Camera Calibration and Distortion Parameters Using Deep Learning and Synthetic Data

TL;DR

This work tackles the challenge of estimating intrinsic camera parameters, including distortion coefficients, from a single image. It introduces the Deep-BrownConrady approach, a ResNet-based regression model trained primarily on a synthetic CPS dataset created with the AILiveSim simulator, incorporating the Brown-Conrady distortion model and image-size information to generalize to real-world data. Key contributions include synthetic-data generation for robust calibration, an architecture that predicts $k_1,k_2,k_3,p_1,p_2,c_x,c_y,FOV$, and empirical evidence showing strong generalization to KITTI and improved distortion correction over traditional multi-view calibration methods. The results demonstrate practical applicability for autonomous driving and robotics where calibration targets are unavailable, motivating further exploration of feature-rich or graph-based extensions and broader camera models.

Abstract

This research addresses the challenge of camera calibration and distortion parameter prediction from a single image using deep learning models. The main contributions of this work are: (1) demonstrating that a deep learning model, trained on a mix of real and synthetic images, can accurately predict camera and lens parameters from a single image, and (2) developing a comprehensive synthetic dataset using the AILiveSim simulation platform. This dataset includes variations in focal length and lens distortion parameters, providing a robust foundation for model training and testing. The training process predominantly relied on these synthetic images, complemented by a small subset of real images, to explore how well models trained on synthetic data can perform calibration tasks on real-world images. Traditional calibration methods require multiple images of a calibration object from various orientations, which is often not feasible due to the lack of such images in publicly available datasets. A deep learning network based on the ResNet architecture was trained on this synthetic dataset to predict camera calibration parameters following the Brown-Conrady lens model. The ResNet architecture, adapted for regression tasks, is capable of predicting continuous values essential for accurate camera calibration in applications such as autonomous driving, robotics, and augmented reality. Keywords: Camera calibration, distortion, synthetic data, deep learning, residual networks (ResNet), AILiveSim, horizontal field-of-view, principal point, Brown-Conrady Model.

Figures (5)

• Figure 1: Projection of point P onto the image plane using the Pinhole Camera Model sturm2011camera.
• Figure 3: Distortion effects on a city scene from AILiveSim with different H-FOV settings. (a) 90° H-FOV with $k_1 = 0.25$. (b) 150° H-FOV with $k_1 = 0.25$.
• Figure 4: Architecture of the DBC v3 model, based on the ResNet50 framework.
• Figure 5: Normalized pixel-wise errors along top, middle, and bottom horizontal lines for DBC v1, DBC v2, and DBC v3. Each graph shows normalized pixel-wise errors across x-coordinates for all three models.
• Figure 6: Visualization of the distortion and undistortion process. (a) Original line image, (b) Image distorted using true distortion parameters, (c) Undistorted image using predicted parameters from DBC v1, (d) Undistorted image using DBC v2, (e) Undistorted image using DBC v3.