JIGGLE: An Active Sensing Framework for Boundary Parameters Estimation in Deformable Surgical Environments

TL;DR

JIGGLE addresses the challenge of identifying boundary attachments in deformable surgical environments under occlusions and topology changes by fusing a differentiable XPBD soft-body simulator with an Extended Kalman Filter. It introduces an online active-sensing loop that selects actions to maximize information gain while enforcing safety constraints, and it demonstrates robustness to real-tissue data and topology changes like cutting and suturing. The approach combines online probabilistic state estimation, joint estimation of boundary parameters and tissue state, and two scalable controllers (local gradient and large-step sampling) to achieve significant entropy reduction and improved boundary detection compared to baselines. The work shows practical potential for safer, more autonomous deformable-tissue manipulation and provides a foundation for extending to heterogeneous tissues and broader deformable-object manipulation tasks.

Abstract

Surgical automation can improve the accessibility and consistency of life saving procedures. Most surgeries require separating layers of tissue to access the surgical site, and suturing to reattach incisions. These tasks involve deformable manipulation to safely identify and alter tissue attachment (boundary) topology. Due to poor visual acuity and frequent occlusions, surgeons tend to carefully manipulate the tissue in ways that enable inference of the tissue's attachment points without causing unsafe tearing. In a similar fashion, we propose JIGGLE, a framework for estimation and interactive sensing of unknown boundary parameters in deformable surgical environments. This framework has two key components: (1) a probabilistic estimation to identify the current attachment points, achieved by integrating a differentiable soft-body simulator with an extended Kalman filter (EKF), and (2) an optimization-based active control pipeline that generates actions to maximize information gain of the tissue attachments, while simultaneously minimizing safety costs. The robustness of our estimation approach is demonstrated through experiments with real animal tissue, where we infer sutured attachment points using stereo endoscope observations. We also demonstrate the capabilities of our method in handling complex topological changes such as cutting and suturing.

Figures (13)

• Figure 1: JIGGLE conducts probabilistic estimation of soft tissue attachment points from image data and manipulation of the tissue. In our real-world ex-vivo experiments, a surgical tool manipulates chicken skin that is attached to its environment with sutures, and we use a stereo endoscope as the observation to estimate the boundary parameters in a probabilistic fashion. The estimated boundary, i.e. the suture locations, is shown in purple. Since the boundary parameters are estimated probabilistically, a corresponding confidence metric can be found and is shown in blue. Note that the confidence is only high near the grasping location because that is where the deformation is applied.
• Figure 2: An illustration of the spring boundary and grasping constraints discussed in section \ref{['sec:PBD_simulator']}. Attachment points on the tissue are depicted in red and are modeled using springs of varying stiffness that connect the tissue to the environment. Meanwhile, the grasp is modeled as an infinite-mass virtual particle that is connected via springs to a small neighborhood of particles highlighted in blue.
• Figure 3: The images show the ground truth attachment points in red using spring boundary constraints for the simulation test cases in our experiments.
• Figure 4: Example results from our estimation framework on simulated environments where the order of the grasps is numbered, and the boundary is highlighted in red on the left-most column. The final result from our proposed method is shown in red in the middle column. Finally, the confidence (inverse of variance) of our estimation is shown in blue in the rightmost column. We can see how the variance has decreased in the regions where the trajectories have displaced the tissue from its original state, and the mean estimate has converged close to the reference values.
• Figure 5: Results from our active sensing experiments with 4 different strategies: PMP (yellow) is a baseline, and SL-D (blue), LG-H (green), and LG-D (red) are proposed in this work. Every two rows show one experiment with the entropy and energy plotted in the left-most column, and the images on the right show a collage of the control trajectories being applied from the different active sensing strategies. Note that the colored background on each image corresponds to the active sensing strategy (best viewed in color). The red color on the tissue highlights the reference attachment, and the blue shows the confidence, inverse of variance, of the estimated boundary. The goal of the active sensing strategies is to maximize the confidence, which is measured in entropy, while adhering to safety constraints, which are measured in energy. Overall, SL-D achieves more entropy reduction than all other baselines while keeping a safe boundary energy profile. It also produces the most intricate control point trajectories, such as switching directions and folding. In comparison, local controllers LG-H and LG-D get trapped in local minima, resulting in higher entropy. PMP reduces entropy in the beginning but results in quick safety violations.

Theorems & Definitions (2)

• Proposition 1
• Lemma 1