Estimating continuous data of wrist joint angles using ultrasound images

TL;DR

This work tackles the problem of estimating continuous wrist joint angles from ultrasound images to enable low-cost, low-power human--machine interfaces. It processes ultrasound-driven muscle motion by detecting Shi--Tomasi feature points, tracking them with optical flow to build a coordinate matrix $\mathbf{S}$, and estimates angles via ridge regression with the analytic solution $w=\boldsymbol\theta \mathbf{S}^{\mathrm{T}}(\mathbf{S}\mathbf{S}^{\mathrm{T}}+\lambda \mathbf{I})^{-1}$ and loss $L=\sum_k(\theta_k-\hat{\theta}_k)^2+\frac{1}{2}\lambda\sum_i (w^i)^2$, using $\lambda=10$. The method achieves RMSE $= 1.82 \pm 0.54$ deg and $R^2=0.985 \pm 0.009$ across ten participants, demonstrating high accuracy with a simple, low-cost model suitable for lightweight wearables. The study discusses placement variability and points to future improvements with smaller wearable probes and optimized placement, framing muscle dynamics as a computational resource within a biomechanical reservoir computing context that requires only linear readouts. Overall, the results indicate ultrasound-based continuous joint-angle sensing can substantially improve control of prosthetics, exoskeletons, and human--machine interfaces.

Abstract

Ultrasound imaging has recently been introduced as a sensing interface for joint motion estimation. The use of ultrasound images as an estimation method is expected to improve the control performance of assistive devices and human--machine interfaces. This study aimed to estimate continuous wrist joint angles using ultrasound images. Specifically, in an experiment, joint angle information was obtained during extension--flexion movements, and ultrasound images of the associated muscles were acquired. Using the features obtained from ultrasound images, a multivariate linear regression model was used to estimate the joint angles. The coordinates of the feature points obtained using optical flow from the ultrasound images were used as explanatory variables of the multivariate linear regression model. The model was trained and tested for each trial by each participant to verify the estimation accuracy. The results show that the mean and standard deviation of the estimation accuracy for all trials were root mean square error (RMSE)=1.82 $\pm$ 0.54 deg and coefficient of determination (R2)=0.985 $\pm$ 0.009. Our method achieves a highly accurate estimation of joint angles compared with previous studies using other signals, such as surface electromyography, while the multivariate linear regression model is simple and both computational and model training costs are low.

Figures (4)

• Figure 1: Experimental setup and image processing. (a) Photograph of the experimental setup employed in this study. The wrist joint angle (a-2) and ultrasound images (a-3) of the muscles associated with the movement in the flexion-extension direction of the wrist joint were acquired from human participants (a-1). (b) To determine the initial positions of the feature points, we used the Shi--Tomasi corner detection method for the first frame of the ultrasound images. The red dots in (b-1) are an example of initial positions of feature points obtained using Shi--Tomasi corner detection. The coordinates of each feature point extracted by corner detection were tracked using optical flow to obtain coordinate data. (b-2) shows the example of the optical flow image after several cycles of wrist motion. Here, the series of gray dots indicates the path of the coordinates.
• Figure 2: Representative examples of wrist joint angle estimation results. (a) The red line is the measured value and the blue line is the estimated value. The left side of the vertical gray dotted line is the training data and the right side is the validation data. The RMSE and R2 of this trial were 0.855 deg and 0.997, respectively. (b) Relationship between measured and estimated wrist joint angles in the validation data section. The gray dotted line is the y=x line. The measured and estimated values are on the y=x line, and there is no large hysteresis, so the measured and estimated values are in good agreement. These results results indicate that a highly accurate estimation was achieved in estimating joint angles using ultrasound image.
• Figure 3: RSME results for estimation accuracy for each participant. From this results, we can conclude that the accuracy of the estimation did not decrease substantially depending on the participant.
• Figure 4: The ultrasound images acquired during the experiment of each participant. The anatomical structure of the forearm, such as the thickness of the muscles, differs slightly from participant to participant, and the position of the ultrasound probe during the measurement differs slightly from participant to participant.