Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A New Method in Facial Registration in Clinics Based on Structure Light Images

Pengfei Li, Ziyue Ma, Hong Wang, Juan Deng, Yan Wang, Zhenyu Xu, Feng Yan, Wenjun Tu, Hong Sha

TL;DR

This work tackles the challenge of registering facial depth images for neurosurgical image fusion by introducing a depth-to-depth registration workflow that leverages dlib-based facial keypoints detected on depth frames from a structure-light camera and a CT-derived depth image. Coarse alignment is driven by ICP on keypoint clouds, followed by a fine ICP refinement, yielding a RMSE of approximately $0.996$ mm after refinement and improved processing time compared with traditional methods. The method demonstrates robustness in challenging scenarios and is complemented by the release of libFaceRegistration for broader clinical use, potentially enhancing neurosurgical navigation and robotic guidance. The approach promises more automated, accurate integration of depth data with preoperative imaging, improving detail-rich visualization for surgical planning and execution.

Abstract

Background and Objective: In neurosurgery, fusing clinical images and depth images that can improve the information and details is beneficial to surgery. We found that the registration of face depth images was invalid frequently using existing methods. To abundant traditional image methods with depth information, a method in registering with depth images and traditional clinical images was investigated. Methods: We used the dlib library, a C++ library that could be used in face recognition, and recognized the key points on faces from the structure light camera and CT image. The two key point clouds were registered for coarse registration by the ICP method. Fine registration was finished after coarse registration by the ICP method. Results: RMSE after coarse and fine registration is as low as 0.995913 mm. Compared with traditional methods, it also takes less time. Conclusions: The new method successfully registered the facial depth image from structure light images and CT with a low error, and that would be promising and efficient in clinical application of neurosurgery.

A New Method in Facial Registration in Clinics Based on Structure Light Images

TL;DR

This work tackles the challenge of registering facial depth images for neurosurgical image fusion by introducing a depth-to-depth registration workflow that leverages dlib-based facial keypoints detected on depth frames from a structure-light camera and a CT-derived depth image. Coarse alignment is driven by ICP on keypoint clouds, followed by a fine ICP refinement, yielding a RMSE of approximately mm after refinement and improved processing time compared with traditional methods. The method demonstrates robustness in challenging scenarios and is complemented by the release of libFaceRegistration for broader clinical use, potentially enhancing neurosurgical navigation and robotic guidance. The approach promises more automated, accurate integration of depth data with preoperative imaging, improving detail-rich visualization for surgical planning and execution.

Abstract

Background and Objective: In neurosurgery, fusing clinical images and depth images that can improve the information and details is beneficial to surgery. We found that the registration of face depth images was invalid frequently using existing methods. To abundant traditional image methods with depth information, a method in registering with depth images and traditional clinical images was investigated. Methods: We used the dlib library, a C++ library that could be used in face recognition, and recognized the key points on faces from the structure light camera and CT image. The two key point clouds were registered for coarse registration by the ICP method. Fine registration was finished after coarse registration by the ICP method. Results: RMSE after coarse and fine registration is as low as 0.995913 mm. Compared with traditional methods, it also takes less time. Conclusions: The new method successfully registered the facial depth image from structure light images and CT with a low error, and that would be promising and efficient in clinical application of neurosurgery.
Paper Structure (12 sections, 8 figures, 2 tables)

This paper contains 12 sections, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Materials
  4. Facial key points acquired
  5. Coarse registration and fine registration
  6. Results
  7. Depth images and facial key points acquired
  8. Coarse registration and fine registration
  9. Accuracy and consuming
  10. Discussion
  11. Conclusions
  12. Acknowledgments

Figures (8)

  • Figure 1: The outline of the registration method.
  • Figure 2: The depth images acquired. (a) The depth image from the structure light camera. (b) The depth image from the CT image.
  • Figure 3: The 2D face pictures. (a) The 2D face picture is generated from the depth image of the CT image. (b) The face key points are recognized by the dlib library.
  • Figure 4: The facial key points acquired. (a) The facial key points from the structure light camera. (b) The facial key points from the CT image.
  • Figure 5: The result of coarse registration. (a) The facial key points registration. The red points are from the structure light camera. The white points are from the CT image. (b) The result of coarse registration.
  • ...and 3 more figures