Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robot-Enabled Machine Learning-Based Diagnosis of Gastric Cancer Polyps Using Partial Surface Tactile Imaging

Siddhartha Kapuria, Jeff Bonyun, Yash Kulkarni, Naruhiko Ikoma, Sandeep Chinchali, Farshid Alambeigi

TL;DR

This work tackles the challenge of endoscopic gastric cancer diagnosis under data scarcity by introducing a Vision-based Tactile Sensor (HySenSe) and a robot-assisted data collection pipeline to capture partial surface textures from realistic AGC phantoms. A Dilated ResNet model, trained on a large, synthetically generated tactile texture dataset, achieves high macro-averaged performance (e.g., AUC ≈ $0.999$, Accuracy ≈ $0.967$) and demonstrates robustness across mixed morphologies and partial contact. The approach addresses data distribution biases and the limitations of endoscopic imaging, showing promise for texture-based AGC polyp classification and guiding future clinical validation and Sim2Real transfer. Overall, the combination of VTS, robotic sampling, and a lightweight yet effective CNN architecture provides a feasible path toward texture-aware diagnostic assistance for AGC lesions. "$4\,\text{cm}^2$ coverage, $50\,\mu\text{m}$ texture resolution, and $3\text{ N}$ interaction cap" are among the concrete constraints that the study successfully manages to enable robust learning from partial observations.

Abstract

In this paper, to collectively address the existing limitations on endoscopic diagnosis of Advanced Gastric Cancer (AGC) Tumors, for the first time, we propose (i) utilization and evaluation of our recently developed Vision-based Tactile Sensor (VTS), and (ii) a complementary Machine Learning (ML) algorithm for classifying tumors using their textural features. Leveraging a seven DoF robotic manipulator and unique custom-designed and additively-manufactured realistic AGC tumor phantoms, we demonstrated the advantages of automated data collection using the VTS addressing the problem of data scarcity and biases encountered in traditional ML-based approaches. Our synthetic-data-trained ML model was successfully evaluated and compared with traditional ML models utilizing various statistical metrics even under mixed morphological characteristics and partial sensor contact.

Robot-Enabled Machine Learning-Based Diagnosis of Gastric Cancer Polyps Using Partial Surface Tactile Imaging

TL;DR

This work tackles the challenge of endoscopic gastric cancer diagnosis under data scarcity by introducing a Vision-based Tactile Sensor (HySenSe) and a robot-assisted data collection pipeline to capture partial surface textures from realistic AGC phantoms. A Dilated ResNet model, trained on a large, synthetically generated tactile texture dataset, achieves high macro-averaged performance (e.g., AUC ≈ , Accuracy ≈ ) and demonstrates robustness across mixed morphologies and partial contact. The approach addresses data distribution biases and the limitations of endoscopic imaging, showing promise for texture-based AGC polyp classification and guiding future clinical validation and Sim2Real transfer. Overall, the combination of VTS, robotic sampling, and a lightweight yet effective CNN architecture provides a feasible path toward texture-aware diagnostic assistance for AGC lesions. " coverage, texture resolution, and interaction cap" are among the concrete constraints that the study successfully manages to enable robust learning from partial observations.

Abstract

In this paper, to collectively address the existing limitations on endoscopic diagnosis of Advanced Gastric Cancer (AGC) Tumors, for the first time, we propose (i) utilization and evaluation of our recently developed Vision-based Tactile Sensor (VTS), and (ii) a complementary Machine Learning (ML) algorithm for classifying tumors using their textural features. Leveraging a seven DoF robotic manipulator and unique custom-designed and additively-manufactured realistic AGC tumor phantoms, we demonstrated the advantages of automated data collection using the VTS addressing the problem of data scarcity and biases encountered in traditional ML-based approaches. Our synthetic-data-trained ML model was successfully evaluated and compared with traditional ML models utilizing various statistical metrics even under mixed morphological characteristics and partial sensor contact.
Paper Structure (14 sections, 5 figures, 3 tables)

This paper contains 14 sections, 5 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. MATERIALS AND METHODS
  3. Vision Based Tactile Sensor (VTS)
  4. Realistic Tumor Phantoms
  5. Experimental Setup and Robotic Data Collection
  6. Calibration and Registration Procedures
  7. Dataset and Pre-processing
  8. Deep Learning Model
  9. Evaluation
  10. RESULTS AND DISCUSSION
  11. Hyperparameter Search
  12. Cross Validation
  13. Model comparisons
  14. CONCLUSIONS

Figures (5)

  • Figure 1: Experimental Setup including: (1) KUKA LBR Med 14 R820 (KUKA AG); (2) Raspberry Pi 4 Model B; (3) 3D printed mounting plate for the tumor phantoms; (4) Top view of HySenSe sensor showing all components; (5) Example CAD model of synthetic AGC polyp phantom; (6) Example partial textural image output of AGC tumor phantom. Figure also shows the defined reference frames R: Robot/World, B: Robot Flange, C: Camera, T: Target, and H: HySenSe base.
  • Figure 2: The first column shows the schematics Wang2015ComparingSO of 4 types of AGC tumors under the Borrmann classification. The second column indicates the corresponding real clinical endoscopic images Hosoda2018ReemergingRO. The third and fourth columns show a sample designed CAD model and 3D-printed tumor for each class. Other columns show corresponding VTS outputs.
  • Figure 3: Hyperparameter search for the three models. Each line encodes a particular configuration.
  • Figure 4: Stratified 5-fold cross-validation results for the three models. Average accuracy curves are reported, with the shaded region depicting the standard deviation across folds.
  • Figure 5: Normalized confusion matrices for all three models configured with their corresponding best hyperparameters.