Multi-camera calibration with pattern rigs, including for non-overlapping cameras: CALICO

Abstract

This paper describes CALICO, a method for multi-camera calibration suitable for challenging contexts: stationary and mobile multi-camera systems, cameras without overlapping fields of view, and non-synchronized cameras. Recent approaches are roughly divided into infrastructure- and pattern-based. Infrastructure-based approaches use the scene's features to calibrate, while pattern-based approaches use calibration patterns. Infrastructure-based approaches are not suitable for stationary camera systems, and pattern-based approaches may constrain camera placement because shared fields of view or extremely large patterns are required. CALICO is a pattern-based approach, where the multi-calibration problem is formulated using rigidity constraints between patterns and cameras. We use a {\it pattern rig}: several patterns rigidly attached to each other or some structure. We express the calibration problem as that of algebraic and reprojection error minimization problems. Simulated and real experiments demonstrate the method in a variety of settings. CALICO compared favorably to Kalibr. Mean reconstruction accuracy error was $\le 0.71$ mm for real camera rigs, and $\le 1.11$ for simulated camera rigs. Code and data releases are available at \cite{tabb_amy_2019_3520866} and \url{https://github.com/amy-tabb/calico}.

Figures (4)

• Figure 1: Best viewed in color. Cameras and patterns for two types of real experiments, box and robot. The top row, (a) is an arrangement of cameras mounted on the periphery of a box where cameras point towards the box's interior; the bottom row, (b) and (c) are multi-camera camera rigs where cameras point away from each other. In the top figure, there are 12 cameras. The pattern rig is moved within the workspace, and cameras acquire an image for every move. The bottom row shows a multicamera system, in (b) only two cameras are used while in (c) all four cameras are used. In the multicamera system, the multicamera system is moved and the patterns are stationary. Each of these experiment types are challenging to calibrate using existing methods. In the box type, cameras share a common field of view but cameras on one side of the box may not be able to view the same planar pattern as cameras on another side. In the robot configuration, there may be no shared field of view between cameras. CALICO approximately solves the set of constraints resulting from the camera, pattern, and time relationships in such datasets.
• Figure 2: Examples of simulated datasets. Arrangements of cameras for four of nine simulated datasets. (a) shows cameras as pyramids arranged on the edges of a square, and a pattern rig of four charuco patterns at one time step, a box type experiment. (b) shows cameras as pyramids and the placement of April Tag calibration patterns over all time steps, a robot type experiment. (c) has cameras from dataset Sim-7-april and one April Tag calibration pattern, another robot type experiment. (d) has the two cameras from Sim-8-april and a very large and long April Tag calibration pattern; there is a barrier so the cameras each view mutually exclusive portions of the pattern.
• Figure 3: Graphical representation of one constraint in $\mathop{\mathrm{\mathbb{CS}}}\nolimits$.
• Figure 4: The interaction graph for Dataset Mult-1. Edges connected to Camera ${}^{0}\mathbf{C}$ are blue, those connected to Camera ${}^{1}\mathbf{C}$ are light grey. $\mathbf{T}^{0} = t^*$ and ${}^{0}\mathbf{P} = p^*$ in this dataset.