Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RT-DEMT: A hybrid real-time acupoint detection model combining mamba and transformer

Shilong Yang, Qi Zang, Chulong Zhang, Lingfeng Huang, Yaoqin Xie

TL;DR

RT-DEMT addresses fast and precise acupoint localization by integrating a Mamba-based state-space backbone with a Transformer-based DETR-like decoder to enable real-time global information integration. It introduces residual likelihood estimation to bypass upsampling and employs an efficient hybrid encoder for multi-scale feature fusion. On a private back acupoint dataset, it achieves $EPE = 7.792$ px and $T_{avg} = 10.05$ ms per localization, roughly 14% faster and more accurate than the next-best method. The approach shows strong potential for automated acupuncture robotics and can be extended to broader pose-estimation tasks.

Abstract

Traditional Chinese acupuncture methods often face controversy in clinical practice due to their high subjectivity. Additionally, current intelligent-assisted acupuncture systems have two major limitations: slow acupoint localization speed and low accuracy. To address these limitations, a new method leverages the excellent inference efficiency of the state-space model Mamba, while retaining the advantages of the attention mechanism in the traditional DETR architecture, to achieve efficient global information integration and provide high-quality feature information for acupoint localization tasks. Furthermore, by employing the concept of residual likelihood estimation, it eliminates the need for complex upsampling processes, thereby accelerating the acupoint localization task. Our method achieved state-of-the-art (SOTA) accuracy on a private dataset of acupoints on the human back, with an average Euclidean distance pixel error (EPE) of 7.792 and an average time consumption of 10.05 milliseconds per localization task. Compared to the second-best algorithm, our method improved both accuracy and speed by approximately 14\%. This significant advancement not only enhances the efficacy of acupuncture treatment but also demonstrates the commercial potential of automated acupuncture robot systems. Access to our method is available at https://github.com/Sohyu1/RT-DEMT

RT-DEMT: A hybrid real-time acupoint detection model combining mamba and transformer

TL;DR

RT-DEMT addresses fast and precise acupoint localization by integrating a Mamba-based state-space backbone with a Transformer-based DETR-like decoder to enable real-time global information integration. It introduces residual likelihood estimation to bypass upsampling and employs an efficient hybrid encoder for multi-scale feature fusion. On a private back acupoint dataset, it achieves px and ms per localization, roughly 14% faster and more accurate than the next-best method. The approach shows strong potential for automated acupuncture robotics and can be extended to broader pose-estimation tasks.

Abstract

Traditional Chinese acupuncture methods often face controversy in clinical practice due to their high subjectivity. Additionally, current intelligent-assisted acupuncture systems have two major limitations: slow acupoint localization speed and low accuracy. To address these limitations, a new method leverages the excellent inference efficiency of the state-space model Mamba, while retaining the advantages of the attention mechanism in the traditional DETR architecture, to achieve efficient global information integration and provide high-quality feature information for acupoint localization tasks. Furthermore, by employing the concept of residual likelihood estimation, it eliminates the need for complex upsampling processes, thereby accelerating the acupoint localization task. Our method achieved state-of-the-art (SOTA) accuracy on a private dataset of acupoints on the human back, with an average Euclidean distance pixel error (EPE) of 7.792 and an average time consumption of 10.05 milliseconds per localization task. Compared to the second-best algorithm, our method improved both accuracy and speed by approximately 14\%. This significant advancement not only enhances the efficacy of acupuncture treatment but also demonstrates the commercial potential of automated acupuncture robot systems. Access to our method is available at https://github.com/Sohyu1/RT-DEMT

Paper Structure

This paper contains 3 sections, 8 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Key Points Detection Task
  3. evaluating indicator

Figures (3)

  • Figure 1: our network structure
  • Figure 2: The green dots represent the gold standard locations of acupoints in the test set, while the red dots indicate the predicted positions of acupoints by the model.
  • Figure 3: The top-left image shows the standard acupoint locations on the back according to national standards, while the top-right image depicts the predicted acupoint locations from our model. The two images below represent traditional pose estimation tasks.The figure clearly shows that the number of points detected for acupoints significantly exceeds that for posture detection. This not only indicates the complexity of our task but also highlights the potential for future applications in fields such as VR and digital humans.