Proprioceptive and Exteroceptive Information Perception in a Fabric Soft Robotic Arm via Physical Reservoir Computing with minimal training data

TL;DR

The method of exploring efficient readout training methods presented here could be extended to other soft robotic systems to maximize their perception capabilities and reveals that balanced linear and nonlinear body dynamics are critical for the physical reservoir to accomplish complex proprioceptive and exteroceptive information perception tasks.

Abstract

Over the past decades, we have witnessed a rapid emergence of soft and reconfigurable robots thanks to their capability to interact safely with humans and adapt to complex environments. However, their softness makes accurate control very challenging. High-fidelity sensing is critical in improving control performance, especially posture and contact estimation. To this end, traditional camera-based sensors and load cells have limited portability and accuracy, and they will inevitably increase the robot's cost and weight. In this study, instead of using specialized sensors, we only collect distributed pressure data inside a pneumatics-driven soft arm and apply the physical reservoir computing principle to simultaneously predict its kinematic posture (i.e., bending angle) and payload status (i.e., payload mass). Our results show that, with careful readout training, one can obtain accurate bending angle and payload mass predictions via simple, weighted linear summations of pressure readings. In addition, our comparative analysis shows that, to guarantee low prediction errors within 10\%, bending angle prediction requires less training data than payload prediction. This result reveals that balanced linear and nonlinear body dynamics are critical for the physical reservoir to accomplish complex proprioceptive and exteroceptive information perception tasks. Finally, the method of exploring the most efficient readout training methods presented in this paper could be extended to other soft robotic systems to maximize their perception capabilities.

Figures (6)

• Figure 1: A "big-picture" comparison of two different approaches for pneumatic soft robot sensing (a) One can embed different kinds of sensors into the soft robotic body and process their readings --- using machine learning, for example --- to obtain proprioceptive and exteroceptive information. In this case, pressure sensors are typically used for actuation control only. (b) One can also treat the robotic body as a reservoir computer (i.e., a "physical neural network") and obtain complex information by simple, weighted linear summations of its pressure sensor readings. The soft robotic arm figure is adopted from nguyen2019soft.
• Figure 2: Setting up the robotic arm as a physical reservoir computer. (a-b) In this study, we use a fabric-based robotic arm segment as the physical testbed, which has three columns of pressurized pillows. (b-c) Working principle of the soft robotic arm reservoir. There are two different inputs to the reservoir: actuation pressure $P^{(i)}$ and end payloads $M^{(j)}$. Seven pressure sensors are distributed throughout the sensing column, and their readings (aka. the reservoir states) will be plugged into weighted linear summation for readout training and prediction. (d) Measured actuation pressures $s_{in}(t)$ sent to the robotic arm. (e) The corresponding robot bending angle captured by the motion capture system (without any payload). (f) Pressure sensor readings from actuation pressure input $P^{(1)}$and $P^{(7)}$ under payload input $M^{(1)}$, these data constitute pressure state vector $\mathbf{S}^{(1,1)}(t)$and $\mathbf{S}^{(7,1)}(t)$.
• Figure 3: Predicting the bending angle under seven actuation pressure profiles $P^{(1)} \ldots P^{(7)}$ and no payloads. (a) Prediction errors in the bending angle under each input pressure profile using different set-ups for readout training. Each row in this matrix corresponds to a unique combination of input conditions for readout training (marked by small triangles). For example, the second row shows the prediction errors from readout training with $P^{(1)}$ and $P^{(7)}$ data, and the last row shows the prediction errors with training data from all seven input conditions. Each column of this matrix shows the arm reservoir's prediction error corresponding to the seven actuation pressure profiles. (b) Detailed training setup using two input conditions $P^{(1)}$ and $P^{(7)}$. (c) The corresponding predictions, where the solid lines are ground truth from the mo-cap system, and dashed lines are the reservoir's predictions. (d) Correlation matrix for 7 different sensor readings under $P^{(1)}$. (e) Correlation matrix of sensor data $s_7(t)$ under 7 different input conditions. (f) The averaged prediction errors based on different choices of two input conditions for reading training. (g) The corresponding prediction results under different pairs for training setup corresponding to the first row in (f).
• Figure 4: Predicting the existence and mass of the end payloads. (a) Training and prediction results of the payload detection task to determine whether a payload exists. (b) Pictures of the robotic arm's deformation under two payloads. (c) Prediction error (in %) of payload mass based on different combinations of input conditions for readout training. Each row in this matrix corresponds to a unique selection of the input conditions (marked by the small triangles). For example, the second row shows the results from training with data from $m^{(2)}$ and $m^{(7)}$ inputs, and the last row with all six payloads. Each column of this matrix shows the arm reservoir's prediction errors of a particular payload. (d) Detailed readout training setup and the training outputs using all non-zero payload mass input conditions. This setup corresponds to the last row in the colormap. (e) The testing data corresponding to the best readout training setup. Here, the thick solid lines are the true payload mass, and the thin lines are the reservoir's prediction. (f) The reservoir's prediction for the 6 different payloads using different readout training setups. This is the same data as shown in the colormap but presented in a different way. (g) Correlation matrix of sensor data $s_7(t)$ under 7 different payloads.
• Figure 5: Arm reservoir's prediction accuracy with reduced training data or reduced number of sensors. (a) Averaged prediction errors for bending posture tasks using different amounts of data samples for readout training. The seven lines correspond to the 7 different actuation pressure inputs. (b) Averaged error for payload mass prediction using different amounts of data samples for readout training. The six lines represent the six non-zero input masses. (c) The overall prediction errors with reduced numbers of sensors for training and prediction. The number of sensors decreases from 6 to 2 (left to right). (d) Sample results of bending angle prediction with fewer sensors. (e) The corresponding percentage portion of readout weights for each sensor involved. Each row adds up to 100%. (e-h) Similar studies on reduced sensors for the payload mass prediction task.