Neural-Augmented Kelvinlet for Real-Time Soft Tissue Deformation Modeling

TL;DR

This work tackles real-time soft tissue deformation under tool interaction by marrying physics-based priors with data-driven learning. It introduces Kelvinlets as closed-form elastic priors for the Navier–Cauchy equations and integrates them through residual learning and regularization to improve accuracy, generalization, and physical plausibility while preserving latency. A large FEM liver deformation dataset (20,800 solves) supports diverse single- and multi-grasper scenarios, and the authors demonstrate consistent performance gains across multiple neural backbones, with near real-time inference. The approach yields substantial improvements in deformation fidelity and temporal stability, enabling principled, computationally efficient, physics-aware surgical AI and simulation pipelines, and the authors provide GPU-accelerated code and data for broad use.

Abstract

Accurate and efficient modeling of soft-tissue interactions is fundamental for advancing surgical simulation, surgical robotics, and model-based surgical automation. To achieve real-time latency, classical Finite Element Method (FEM) solvers are often replaced with neural approximations; however, naively training such models in a fully data-driven manner without incorporating physical priors frequently leads to poor generalization and physically implausible predictions. We present a novel physics-informed neural simulation framework that enables real-time prediction of soft-tissue deformations under complex single- and multi-grasper interactions. Our approach integrates Kelvinlet-based analytical priors with large-scale FEM data, capturing both linear and nonlinear tissue responses. This hybrid design improves predictive accuracy and physical plausibility across diverse neural architectures while maintaining the low-latency performance required for interactive applications. We validate our method on challenging surgical manipulation tasks involving standard laparoscopic grasping tools, demonstrating substantial improvements in deformation fidelity and temporal stability over existing baselines. These results establish Kelvinlet-augmented learning as a principled and computationally efficient paradigm for real-time, physics-aware soft-tissue simulation in surgical AI.

Figures (4)

• Figure 1: Overview of the large-scale FEM data generation pipeline. A patient’s abdominal CT scan is first anatomically segmented (left), followed by the generation of a 3D linear tetrahedral mesh (middle). Diverse interactions are then applied to the mesh, and FEM simulations are performed to produce training data for ML models (right). The right panel shows the norms of the input (interactions) and the resulting output displacements (predicted global effects).
• Figure 2: Overview of the two proposed training frameworks. Method 1 (Left): Residual learning predicts only the deviation from the Kelvinlet solution, reducing learning complexity by leveraging the physics-based analytical solution as an initialization. Method 2 (Right): Kelvinlet-based regularization combines FEM-supervised learning with a secondary loss enforcing consistency with Kelvinlet priors. The heatmap depicts the displacement norms.
• Figure 3: Qualitative comparison of deformation predictions for the best (PointNet) and worst (GraphGPS) models in linear regimes.
• Figure 4: Log-scaled inference time vs. accuracy (DCM%) for various models in linear and nonlinear simulations. Transparent markers denote base models, while solid markers indicate Kelvinlet-regularized versions. Black-edged markers represent multi-grasper cases, while edge-free markers denote single-grasper cases. FEM and Kelvinlet baselines are included for reference.