3D Cal: An Open-Source Software Library for Calibrating Tactile Sensors

TL;DR

3D Cal addresses the calibration bottleneck in tactile sensing by converting low-cost 3D printers into automated probing devices that rapidly collect labeled data for calibrating vision-based tactile sensors. It introduces TouchNet, a lightweight 9-layer CNN that maps RGB tactile images plus position embeddings to surface gradients, with Poisson integration producing depth maps, and enables fast inference ($30$ ms). Through data collection and ablation studies on DIGIT and GelSight Mini, the work provides concrete guidelines (≈$240$ coordinates) for achieving robust calibration and demonstrates depth-map reconstruction on unseen objects with moderate per-pixel error (typically tens to a few hundred micrometers). The open-source library, pre-trained models, and large-scale tactile datasets are poised to accelerate tactile sensing research, enable sensor deployment, and promote transfer learning and broader ecosystem development.

Abstract

Tactile sensing plays a key role in enabling dexterous and reliable robotic manipulation, but realizing this capability requires substantial calibration to convert raw sensor readings into physically meaningful quantities. Despite its near-universal necessity, the calibration process remains ad hoc and labor-intensive. Here, we introduce 3D Cal, an open-source library that transforms a low-cost 3D printer into an automated probing device capable of generating large volumes of labeled training data for tactile sensor calibration. We demonstrate the utility of 3D Cal by calibrating two commercially available vision-based tactile sensors, DIGIT and GelSight Mini, to reconstruct high-quality depth maps using the collected data and a custom convolutional neural network. In addition, we perform a data ablation study to determine how much data is needed for accurate calibration, providing practical guidelines for researchers working with these specific sensors, and we benchmark the trained models on previously unseen objects to evaluate calibration accuracy and generalization performance. By automating tactile sensor calibration, 3D Cal can accelerate tactile sensing research, simplify sensor deployment, and promote the practical integration of tactile sensing in robotic platforms.

Figures (5)

• Figure 1: Overview of 3D Cal library. (A) Users first 3D print a sensor base on the print bed. (B) The tactile sensor is then inserted into the base with a slide fit. (C) Next, the 3D printer nozzle is affixed with a probe tip. (D) An automated program then uses the 3D printer to probe the sensor and collect labeled calibration data. (E) The calibration data is then used to train or fine-tune a machine learning model. For vision-based tactile sensors, we include our TouchNet model, which employs a 9-layer convolutional architecture to convert an RGB image, appended with a 2-channel x,y coordinate embedding, into a surface greadient map. (F) The trained models are then used to predict calibration targets, which for TouchNet is an indentation depth map.
• Figure 2: Results of training data ablation study. (A) Sensors were probed along an $m \times n$ grid with d = 0.5 mm spacing. 20% of the coordinates were reserved for validation (red dots) while the remaining coordinates were used for training (black dots). Models were trained on $P = 80\%, 40\%, 20\%, 10\%, 5\%,$ and $1\%$ of the total coordinates. (B) Model loss when trained on different percentages, $P$, of the probed coordinates. To account for differences in the number of batches per epoch, models were trained for $N = 60 \times (\frac{80\%}{P})$ epochs. (C) Reconstructed depth maps (pill-shaped test object) for models trained on different percentages, $P$, of total coordinates.
• Figure 3: Spatial distribution of reconstruction accuracy. (A) The mean squared error (MSE) of the output gradients for coordinates in the validation set (shaded circles) using TouchNet models trained on different percentages, $P$, of the probed coordinates. The MSE tended to be higher in regions with fewer training coordinates (red X's). (B) Kernel density estimates (KDE) of the MSE values (bin width: 0.0015, dashed lines: mean). These distributions tended to converge towards one another for higher values of $P$. (C) Standard deviation ($\sigma$) of MSE distributions for different P values (DIGIT: black line, GelSight Mini: red line).
• Figure 4: Reconstructed depth maps for DIGIT and GelSight Mini using 3D Cal. (A) Three 3D printed test objects (hemispheres, pill, pawn), all measuring $10\ \mathrm{mm} \times 10\ \mathrm{mm}$, were pushed into the DIGIT and GelSight Mini. The corresponding RGB images on the DIGIT (bottom left) and GelSight Mini (bottom right) are shown below each test object. (B) The TouchNet models trained on $P = 80\%$ of the probed coordinates were used predict the depth maps for both sensors. (C) Ground-truth depth maps were computed from the CAD models and a 2D cross-correlation was performed to line up the ground-truth and predicted depth maps. Cross sections along two perpendicular axes show the predicted depth (orange line) relative to the ground-truth depth (blue line).
• Figure 5: Distributions of pixelwise depth map errors. (A) Violin plots of the pixelwise depth map error distributions where the ground-truth depth was equal to zero (Type 1 Error). Type 1 error distributions were truncated at the 95th percentile for clarity (DIGIT: red, GelSight Mini: blue). (B) Violin plots of the pixelwise depth map error distributions where the ground-truth depth was non-zero (Type 2 Error).