SurgFormer: Scalable Learning of Organ Deformation with Resection Support and Real-Time Inference

TL;DR

This work introduces SurgFormer, a multiresolution gated transformer for data driven soft tissue simulation on volumetric meshes and is the first learned volumetric surrogate setting to study XFEM supervised cut conditioned deformation within the same volumetric pipeline as standard deformation prediction.

Abstract

We introduce SurgFormer, a multiresolution gated transformer for data driven soft tissue simulation on volumetric meshes. High fidelity biomechanical solvers are often too costly for interactive use, so we train SurgFormer on solver generated data to predict nodewise displacement fields at near real time rates. SurgFormer builds a fixed mesh hierarchy and applies repeated multibranch blocks that combine local message passing, coarse global self attention, and pointwise feedforward updates, fused by learned per node, per channel gates to adaptively integrate local and long range information while remaining scalable on large meshes. For cut conditioned simulation, resection information is encoded as a learned cut embedding and provided as an additional input, enabling a unified model for both standard deformation prediction and topology altering cases. We also introduce two surgical simulation datasets generated under a unified protocol with XFEM based supervision: a cholecystectomy resection dataset and an appendectomy manipulation and resection dataset with cut and uncut cases. To our knowledge, this is the first learned volumetric surrogate setting to study XFEM supervised cut conditioned deformation within the same volumetric pipeline as standard deformation prediction. Across diverse baselines, SurgFormer achieves strong accuracy with favorable efficiency, making it a practical backbone for both tasks. {Code, data, and project page: \href{https://mint-vu.github.io/SurgFormer/}{available here}}

Figures (4)

• Figure 1: SurgFormer enables real-time, anatomically plausible soft-tissue simulation for minimally invasive surgery, jointly modeling tool-driven deformation and topology-changing resection. We demonstrate cholecystectomy (top) and appendectomy (bottom); each row shows the input, predicted deformation under tool interaction, and resulting resection state, highlighting geometric fidelity and visual realism.
• Figure 2: XFEM based data generation pipeline for surgical tissue mechanics. CT volumes are segmented to obtain organ geometries, converted to solid models, and tetrahedralized into volumetric meshes. Procedure-specific boundary conditions, tool interaction flags, deleted-vertex indicators, and level-set fields are then defined to represent deformation and resection states. Quasistatic linear XFEM is solved on the active domain to produce full-field displacement targets.
• Figure 3: Overview of the proposed multi resolution graph transformer for organ deformation. Simulator input signals are encoded by an Adapter and processed through $L$ hierarchical levels using farthest point sampling with pooling, where each level applies LayerNorm followed by parallel local graph attention, global multi head attention, and feed forward transformations fused by a learned gate. A symmetric decoder upsamples via unpooling and refines features at each level, using levelwise skip connections from the encoder, and a final regressor maps the finest level features to the output.
• Figure :