Looking Together $\neq$ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and Instruction

TL;DR

This paper investigates how surgeons’ visual attention can be shared and augmented to improve intraoperative coordination and teaching during laparoscopic surgery. Using interviews with 14 surgeons and observations of 8 Lap Chole procedures, it reveals diverse visual needs across training levels, challenges in achieving visual alignment, and cautious optimism about gaze-based guidance for both real-time instruction and post-operative reflection. The authors propose design implications such as on-demand gaze sharing, AR overlays, and semantic gaze visualizations that emphasize anatomical landmarks and decision contexts, aiming to enhance teaching, learning, and safety. The work highlights the potential of gaze data to support residents’ professional vision while recognizing ethical considerations and the need for context-aware, user-controlled visualization in high-stakes surgical environments.

Abstract

Shared gaze visualizations have been found to enhance collaboration and communication outcomes in diverse HCI scenarios including computer supported collaborative work and learning contexts. Given the importance of gaze in surgery operations, especially when a surgeon trainer and trainee need to coordinate their actions, research on the use of gaze to facilitate intra-operative coordination and instruction has been limited and shows mixed implications. We performed a field observation of 8 surgeries and an interview study with 14 surgeons to understand their visual needs during operations, informing ways to leverage and augment gaze to enhance intra-operative coordination and instruction. We found that trainees have varying needs in receiving visual guidance which are often unfulfilled by the trainers' instructions. It is critical for surgeons to control the timing of the gaze-based visualizations and effectively interpret gaze data. We suggest overlay technologies, e.g., gaze-based summaries and depth sensing, to augment raw gaze in support of surgical coordination and instruction.

Figures (1)

• Figure 1: A photo taken by the research team in one of the 8 surgeries we observed, which showed the data collection setup in the observations. Both attending and resident surgeons wore eye-tracking glasses, Tobii Pro Eye Glasses 2 with audio recording capabilities. Using eye-tracking glasses in the observations serves two goals. First, it allows us to record multi-modal data, including camera views from the eye-trackers, and the conversations between the surgeons. This is critical for interpreting the observation data since with audio data alone, it is insufficient for the researchers to fully understand the context. Second, it serves as an internal feasibility check of collecting eye gaze data while surgeons are performing the operation. One of the lead authors was present in the operation through all 8 surgeries.