Self-Supervised Uncalibrated Multi-View Video Anonymization in the Operating Room

TL;DR

This work tackles privacy-preserving analysis of operating-room video by delivering a self-supervised, annotation-free framework for uncalibrated multi-view anonymization. It introduces a two-stage pipeline that first detects whole bodies across multiple views with temporal and geometric context, and then estimates whole-body pose for precise anonymization, all while iteratively refining detectors via pseudo labels. The approach combines tracking, uncalibrated multi-view association, and self-supervised domain adaptation to achieve high recall (exceeding 97%) on both simulated and real surgeries, and demonstrates real-time capability by training a detector on pseudo labels. The method significantly reduces manual review time for anonymization and provides practical applicability for scalable, privacy-preserving OR video research.

Abstract

Privacy preservation is a prerequisite for using video data in Operating Room (OR) research. Effective anonymization relies on the exhaustive localization of every individual; even a single missed detection necessitates extensive manual correction. However, existing approaches face two critical scalability bottlenecks: (1) they usually require manual annotations of each new clinical site for high accuracy; (2) while multi-camera setups have been widely adopted to address single-view ambiguity, camera calibration is typically required whenever cameras are repositioned. To address these problems, we propose a novel self-supervised multi-view video anonymization framework consisting of whole-body person detection and whole-body pose estimation, without annotation or camera calibration. Our core strategy is to enhance the single-view detector by "retrieving" false negatives using temporal and multi-view context, and conducting self-supervised domain adaptation. We first run an off-the-shelf whole-body person detector in each view with a low-score threshold to gather candidate detections. Then, we retrieve the low-score false negatives that exhibit consistency with the high-score detections via tracking and self-supervised uncalibrated multi-view association. These recovered detections serve as pseudo labels to iteratively fine-tune the whole-body detector. Finally, we apply whole-body pose estimation on each detected person, and fine-tune the pose model using its own high-score predictions. Experiments on the 4D-OR dataset of simulated surgeries and our dataset of real surgeries show the effectiveness of our approach achieving over 97% recall. Moreover, we train a real-time whole-body detector using our pseudo labels, achieving comparable performance and highlighting our method's practical applicability. Code is available at https://github.com/CAMMA-public/OR_anonymization.

Figures (6)

• Figure 1: Examples of robust whole-body detection in the operating room using temporal and multi-view context. In the current frame, the detector fails to detect any instance. Using a tracker and detections from previous frame, we find one clinician; using multi-view person association and detections from another view, we find another clinician and the patient.
• Figure 2: Framework of our two-stage approach, including whole-body detection and whole-body pose estimation. During training, we apply iterative domain adaptation for whole-body detection: in each round, (a) we use a whole-body detector to generate detections by applying a low-score threshold; (b) we use tracking to obtain tracklets; (c) we use tracklets and multi-view association to find missing detections; (d) we merge tracklets and association results as augmented detections; (e) we fine-tune the detector using augmented detections as pseudo labels. For the whole-body pose detector, we (f) predict poses and (g) fine-tune the model using the high-score predictions on the augmented whole-body detections.
• Figure 3: Framework of the self-supervised multi-view association approach chen2025learning (FC = Fully-Connected, LN = Layer Normalization, Re-ID = Re-IDentification). For each anchor image, we construct a triplet by selecting synchronized and non-synchronized images in a different view. Then, we encode each detected person's appearance features using a person Re-ID model, map their geometric information to a unified geometric feature space using positional encodings and learnable camera embeddings, and then decode the original 2d position. Finally, we use encoded features to conduct instance association along with triplet-based metric learning. Best viewed in color.
• Figure 4: Comparison of different ways to generate pseudo labels: (1) setting the score threshold to 0.6 will introduce false negatives (missed detections); (2) setting the score threshold to 0.1 will introduce false positives; (3) our strategy combining tracking and multi-view person association obtains pseudo labels of the best quality.
• Figure 5: Examples of different video anonymization approaches on our dataset of real surgeries. (1) RetinaFace deng2020retinaface fails to detect the faces with masks; (2) Head-YOLOv8 varghese2024yolov8 fails to detect the ones with heavily occluded bodies; (3) P-D-DETR zheng2022progressive (whole-body detection) sets up a strong baseline, but still fails in challenging cases; (4) Iter-Score issenhuth2019face regresses the bounding boxes more accurately, but fails to retrieve the false negatives; (5) Our approach detects all the persons using temporal and multi-view context, and supports flexible anonymization with human pose estimation.