Multimodal Sensing for Robot-Assisted Sub-Tissue Feature Detection in Physiotherapy Palpation

TL;DR

Robotic palpation in soft tissue is hindered by the variability of force signals, which can obscure subsurface features. The authors introduce PhysioVisionFT (PVFT), a compact multimodal sensor that fuses high-resolution visuotactile imaging with a 6-axis force–torque sensor to detect subsurface tendon geometry during physiotherapy palpation. Through fabrication, calibration, and silicone phantom experiments, tactile imaging reliably reveals tendon presence, diameter, depth, crossings, and multiplicity while force data provides essential, safe contact control, achieving stable force tracking under practical loads (e.g., $F_z$ and a RMSE of $7.04\%$ for $25~\text{N}$ commands). This work lays a path toward safer, more repeatable robot-assisted palpation and physiotherapy, with future directions including learning-based closed-loop palpation strategies and direct tendon localization from multimodal sensor data.

Abstract

Robotic palpation relies on force sensing, but force signals in soft-tissue environments are variable and cannot reliably reveal subtle subsurface features. We present a compact multimodal sensor that integrates high-resolution vision-based tactile imaging with a 6-axis force-torque sensor. In experiments on silicone phantoms with diverse subsurface tendon geometries, force signals alone frequently produce ambiguous responses, while tactile images reveal clear structural differences in presence, diameter, depth, crossings, and multiplicity. Yet accurate force tracking remains essential for maintaining safe, consistent contact during physiotherapeutic interaction. Preliminary results show that combining tactile and force modalities enables robust subsurface feature detection and controlled robotic palpation.

Figures (9)

• Figure 1: The PhysioVisionFT (PVFT) concept: a force/torque sensor captures ($F_x$, $F_y$, $F_z$) contact force data; a camera captures high-resolution visuotactile images, enabling safe and controlled contact for physiotherapy.
• Figure 2: Exploded View of the PVFT multimodal sensng tool
• Figure 3: Procedure: (1) The robot descends along Z direction until the normal force reaches 25 N, 35 N or 45 N depending on the trial; (2) end-effector holds position for 1 s; (3) robot translates 120 mm along the +Y direction under position control; (4) a second 1 s holding phase; 5) the end-effector retracts and disengages.
• Figure 4: Typical motion (a) and force (b) plots corresponding to Fig. \ref{['fig: Experimental procedure']}, in this case with a single uniform tendon along the whole length. The robot descends (1) until it reaches 25 N normal force. It then dwells (2) for 1 s and proceeds in the +Y direction (3) at constant Z height. Due to "ploughing" of the surface, there is some variation in $F_z$. The robot stops at (4) and departs in (5). Shaded blue region during step (3) shows the change in normal force per equation \ref{['eq:relative force']}.
• Figure 5: Multimodal sensing response when sliding from tendon to no-tendon regions. (a) Shows tactile images corresponding to regions noted in (b). Roman numerals (i), (ii), (ii) show the corresponding times in position (c) and force (d) plots. As in Fig. \ref{['fig:ploughingonUniTendon']}, shaded region shows $-\Delta F_{z}^{\text{abs}}$. Inset (e) shows normalized change in $F_z$ during step 3.