A Cranial-Feature-Based Registration Scheme for Robotic Micromanipulation Using a Microscopic Stereo Camera System

TL;DR

A microscopic stereo camera system enhanced by the linear model for depth perception and a precise registration scheme is developed for the partially exposed mouse cranial surface, employing a CNN-based constrained and colorized registration strategy for robotic micromanipulation tasks.

Abstract

Biological specimens exhibit significant variations in size and shape, challenging autonomous robotic manipulation. We focus on the mouse skull window creation task to illustrate these challenges. The study introduces a microscopic stereo camera system (MSCS) enhanced by the linear model for depth perception. Alongside this, a precise registration scheme is developed for the partially exposed mouse cranial surface, employing a CNN-based constrained and colorized registration strategy. These methods are integrated with the MSCS for robotic micromanipulation tasks. The MSCS demonstrated a high precision of 0.10 mm $\pm$ 0.02 mm measured in a step height experiment and real-time performance of 30 FPS in 3D reconstruction. The registration scheme proved its precision, with a translational error of 1.13 mm $\pm$ 0.31 mm and a rotational error of 3.38$^{\circ}$ $\pm$ 0.89$^{\circ}$ tested on 105 continuous frames with an average speed of 1.60 FPS. This study presents the application of a MSCS and a novel registration scheme in enhancing the precision and accuracy of robotic micromanipulation in scientific and surgical settings. The innovations presented here offer automation methodology in handling the challenges of microscopic manipulation, paving the way for more accurate, efficient, and less invasive procedures in various fields of microsurgery and scientific research.

Figures (15)

• Figure 1: The thickness distribution on the area to be drilled, which is thin and not uniform.
• Figure 2: The robotic system for microscopic manipulation Marinho2024 and the requirements for the 3D vision. $d_e$ is the effective working distance of the stereo microscope.
• Figure 3: A registration framework for microscopic manipulation, where the left side is the overview of the hardware of the MSCS, and the right side is the developed software in ROS. The diagrams on the right side show the workflow of the registration with each module.
• Figure 4: Illustration of the stereo microscopy configuration. B is the baseline distance between the two cameras. The disparity is the distance between the projected points $p_l(x_l',y_l')$ and $p_r(x_r', y_r')$ in the image planes. The depth of field characterizes the setup's focal range. Frame $\{i\}$ is defined at an effective distance $d_e$ from the origin of Frame $\{o\}$ along the z-axis. $d_e$ is defined at the beginning of the depth of field of the stereo microscope.
• Figure 5: A distorted planar board reconstruction by the stereo microscope, which is similar to CMO distortion.