Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-Quality Virtual Single-Viewpoint Surgical Video: Geometric Autocalibration of Multiple Cameras in Surgical Lights

Yuna Kato, Mariko Isogawa, Shohei Mori, Hideo Saito, Hiroki Kajita, Yoshifumi Takatsume

TL;DR

This work tackles occlusion challenges in open-surgery video capture by a five-camera system mounted on a surgical light. It introduces an automated geometric-autocalibration pipeline that detects camera movement via a misalignment metric $D_t$, updates homographies at timing $t_h$ using a planar proxy, and selects low-occlusion frames to synthesize a stable virtual single-view video. The approach outperforms manual alignment and no-alignment in both qualitative surgeon evaluations and quantitative metrics such as interframe transformation fidelity and frame stability, demonstrating potential for improved surgical observation and education. The method offers practical impact by enabling automated, occlusion-reduced video generation in real clinical settings, with quantified gains and identified avenues for refinement.

Abstract

Occlusion-free video generation is challenging due to surgeons' obstructions in the camera field of view. Prior work has addressed this issue by installing multiple cameras on a surgical light, hoping some cameras will observe the surgical field with less occlusion. However, this special camera setup poses a new imaging challenge since camera configurations can change every time surgeons move the light, and manual image alignment is required. This paper proposes an algorithm to automate this alignment task. The proposed method detects frames where the lighting system moves, realigns them, and selects the camera with the least occlusion. This algorithm results in a stabilized video with less occlusion. Quantitative results show that our method outperforms conventional approaches. A user study involving medical doctors also confirmed the superiority of our method.

High-Quality Virtual Single-Viewpoint Surgical Video: Geometric Autocalibration of Multiple Cameras in Surgical Lights

TL;DR

This work tackles occlusion challenges in open-surgery video capture by a five-camera system mounted on a surgical light. It introduces an automated geometric-autocalibration pipeline that detects camera movement via a misalignment metric , updates homographies at timing using a planar proxy, and selects low-occlusion frames to synthesize a stable virtual single-view video. The approach outperforms manual alignment and no-alignment in both qualitative surgeon evaluations and quantitative metrics such as interframe transformation fidelity and frame stability, demonstrating potential for improved surgical observation and education. The method offers practical impact by enabling automated, occlusion-reduced video generation in real clinical settings, with quantified gains and identified avenues for refinement.

Abstract

Occlusion-free video generation is challenging due to surgeons' obstructions in the camera field of view. Prior work has addressed this issue by installing multiple cameras on a surgical light, hoping some cameras will observe the surgical field with less occlusion. However, this special camera setup poses a new imaging challenge since camera configurations can change every time surgeons move the light, and manual image alignment is required. This paper proposes an algorithm to automate this alignment task. The proposed method detects frames where the lighting system moves, realigns them, and selects the camera with the least occlusion. This algorithm results in a stabilized video with less occlusion. Quantitative results show that our method outperforms conventional approaches. A user study involving medical doctors also confirmed the superiority of our method.

Paper Structure

This paper contains 14 sections, 6 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Camera Movement Detection
  4. Detecting the Timing for Obtaining a Homography Matrix
  5. Experiments and Results
  6. Dataset.
  7. Implementation Details.
  8. Virtual Single-View Video Generation.
  9. Comparison with Conventional Methods.
  10. Qualitative Evaluation
  11. Quantitative Evaluation
  12. Conclusion and Discussion
  13. Limitations.
  14. Acknowledgements

Figures (5)

  • Figure 1: Surgical light with multiple cameras. The unit consists of five cameras (left), each of which is surrounded by multiple light sources (right).
  • Figure 2: Overview of the proposed method.
  • Figure 3: Video frames after auto-alignment.
  • Figure 4: Video frames obtained using the three methods. The red lines indicate the position and orientation of the instep of patient's foot.
  • Figure 5: Results of the subjective evaluation experiment.