HybridCollab: Unifying In-Person and Remote Collaboration for Cardiovascular Surgical Planning in Mobile Augmented Reality

TL;DR

The paper tackles the problem of limited collaborative flexibility in congenital heart disease planning by enabling cross-location teamwork in mobile AR. It introduces HybridCollab, the first iOS-based system that unifies in-person and remote teams through GameKit-based real-time collaboration and spatially anchored shared heart models. Key features include up to 16 participants, low-latency synchronization, co-located anchoring with Multipeer Connectivity, and tools for virtual annotations, pointing, and omnidirectional slicing. The authors anticipate significant impact on surgical planning efficiency and plan usability studies with CHOA to validate real-world effectiveness.

Abstract

Surgical planning for congenital heart disease traditionally relies on collaborative group examinations of a patient's 3D-printed heart model, a process that lacks flexibility and accessibility. While mobile augmented reality (AR) offers a promising alternative with its portability and familiar interaction gestures, existing solutions limit collaboration to users in the same physical space. We developed HybridCollab, the first iOS AR application that introduces a novel paradigm that enables both in-person and remote medical teams to interact with a shared AR heart model in a single surgical planning session. For example, a team of two doctors in one hospital room can collaborate in real time with another team in a different hospital.Our approach is the first to leverage Apple's GameKit service for surgical planning, ensuring an identical collaborative experience for all participants, regardless of location. Additionally, co-located users can interact with the same anchored heart model in their shared physical space. By bridging the gap between remote and in-person collaboration across medical teams, HybridCollab has the potential for significant real-world impact, streamlining communication and enhancing the effectiveness of surgical planning. Watch the demo: https://youtu.be/hElqJYDuvLM.

Figures (3)

• Figure 1: Example of a 3D-printed heart model.
• Figure 2: HybridCollab users connected in a session.
• Figure 3: Example annotations.