Embodied multi-modal sensing with a soft modular arm powered by physical reservoir computing

TL;DR

The paper addresses the challenge of high-fidelity sensing in soft robots by leveraging physical reservoir computing (PRC) with embedded bending sensors. It demonstrates that a four-module soft modular arm, actuated by shape memory alloys and equipped with 12 bending sensors, can simultaneously predict body posture and extract payload information (weight and orientation) using simple linear readouts trained on reservoir states. The results show high discrimination between similar payloads, robustness to sensor dropouts, and performance gains over visual data in weight estimation and precise classification. This embodied sensing approach reduces hardware and computational complexity, enabling real-time, multi-modal perception and paving the way for adaptive closed-loop control in soft robotics.

Abstract

Soft robots have become increasingly popular for complex manipulation tasks requiring gentle and safe contact. However, their softness makes accurate control challenging, and high-fidelity sensing is a prerequisite to adequate control performance. To this end, many flexible and embedded sensors have been created over the past decade, but they inevitably increase the robot's complexity and stiffness. This study demonstrates a novel approach that uses simple bending strain gauges embedded inside a modular arm to extract complex information regarding its deformation and working conditions. The core idea is based on physical reservoir computing (PRC): A soft body's rich nonlinear dynamic responses, captured by the inter-connected bending sensor network, could be utilized for complex multi-modal sensing with a simple linear regression algorithm. Our results show that the soft modular arm reservoir can accurately predict body posture (bending angle), estimate payload weight, determine payload orientation, and even differentiate two payloads with only minimal difference in weight -- all using minimal digital computing power.

Figures (10)

• Figure 1: Multi-modal sensing strategies with traditional and proposed physical computing approaches. To estimate the body posture, payload weight, and payload orientation, traditional multi-modal sensing will need several kinds of sensors and cameras (a). Moreover, these sensing data will typically be processed with complex matching learning algorithms using digital computers. In our proposed method with physical reservoir computing (b), the soft body dynamics, which are captured by the simplest bending strain gauges, can be used to extract the same information, significantly reducing the overall system complexity
• Figure 2: Schematic view of precise motion tracking with multi-modal sensing capability using physical reservoir computing. The first step is to train multiple groups of readout weights $W^{(n)}, \; n=1\ldots8$ corresponding to different information targets (a). The ultimate goal is to achieve accurate tracking regardless of the payload setup. (b) The operating procedure of the physical reservoir.
• Figure 3: Fabrication of the soft modular robotic arm with four modules. The module manufacturing process consists of two steps: (a) Panel fabrication and (b) assembly with the bending sensor and shape memory alloy. (c) A 3D-printed gripper is attached to the four connected modules. (d) The arm's size can be scaled up or down.
• Figure 4: Schematic view of precise motion tracking with the arm's multi-modal sensing capability enabled by reservoir computing. The arm bends after two SMA columns are heated. The wiggling behavior of arm is captured by sensor network, which serves as the computing kernel for kinematic and payload information perception.
• Figure 5: An example of motion prediction (horizontal displacement) using the robotic arm's sensor network. (a) represents the collected sensor data, while (b) shows the corresponding training and testing results. The five colorful solid lines indicate the actual movements of module 1 through module 4 and the gripper. The first half of the black solid line represents the reservoir output of the arm, corresponding to five tracked points, while the latter half of the dashed line represents the predicted motion during the testing phase.