A Vision-based Sensing Approach for a Spherical Soft Robotic Arm

TL;DR

A vision-based sensing approach for a soft robotic arm made from fabric is presented, leveraging the high-resolution sensory feedback provided by cameras, and the reliability of the sensing approach is demonstrated by using the sensory feedback to control the orientation of the robotic arm in closed-loop.

Abstract

Sensory feedback is essential for the control of soft robotic systems and to enable deployment in a variety of different tasks. Proprioception refers to sensing the robot's own state and is of crucial importance in order to deploy soft robotic systems outside of laboratory environments, i.e. where no external sensing, such as motion capture systems, is available. A vision-based sensing approach for a soft robotic arm made from fabric is presented, leveraging the high-resolution sensory feedback provided by cameras. No mechanical interaction between the sensor and the soft structure is required and consequently, the compliance of the soft system is preserved. The integration of a camera into an inflatable, fabric-based bellow actuator is discussed. Three actuators, each featuring an integrated camera, are used to control the spherical robotic arm and simultaneously provide sensory feedback of the two rotational degrees of freedom. A convolutional neural network architecture predicts the two angles describing the robot's orientation from the camera images. Ground truth data is provided by a motion capture system during the training phase of the supervised learning approach and its evaluation thereafter. The camera-based sensing approach is able to provide estimates of the orientation in real-time with an accuracy of about one degree. The reliability of the sensing approach is demonstrated by using the sensory feedback to control the orientation of the robotic arm in closed-loop.

Figures (9)

• Figure 1: The figure on the left shows the spherical robotic arm used for evaluation of the vision-based sensing approach proposed in this work. The figure on the right shows images from the cameras placed inside three inflatable bellow actuators. The arrangement of the camera images matches a view from the bottom looking upwards. The orientation of the movable link can be observed in certain actuator elongations and deformations that are observed by the internal cameras.
• Figure 2: The orientation parametrization of the spherical robotic arm is shown in the left hand plot. The static link is aligned with the inertial $z$-axis. A positive rotation of the movable link around the inertial $x$-axis is denoted by $\alpha$ and a positive rotation around the inertial $y$-axis is denoted by $\beta$. The top view of the actuator configuration in the inertial coordinate frame is shown in the right hand plot. The three actuators are arranged symmetrically around the inertial $z$-axis, where the actuator A is aligned with the inertial $x$-axis.
• Figure 3: The figure shows a simplified sketch of the cross section of an actuator with the visible area of the camera indicated in red (dashed). Angle connectors are attached to both the top side of the actuator (shown in black in the left of the figure) and bottom (shown in gray in the bottom of the figure). The bottom connector is used to pressurize the actuator and the top connector to align it with the movable link. The inner opening which connects neighboring cushions has a width denoted by $w$ and plays a crucial role in the resulting visible area of the camera. If the opening is sufficiently wide, the majority of the cushions are within the visible area of the camera. The area of the actuator deformation covered by the camera is increased, if the camera is placed with an offset $\eta$ with respect to the center of the inner opening and tilted by an angle $\rho$ with respect to the normal direction.
• Figure 4: (A) The figure shows the camera electronics used in each actuator. The camera and LED board are connected to a to a custom-made adapter board which reroutes the camera pins and powers the LED board. The adapter board includes pins which are used for synchronizing multiple cameras and is powered over the black/red cables. The adapter board is connected to an Arducam USB Camera Shield (UC-425 Rev. C) with USB interface (USB cable not shown). (B) The picture shows the front view of the camera adapter housing the camera and the enclosing LED board. The camera is tilted by an angle of $\unit[25]{^\circ}$ with respect to the normal direction of the adapter plane. The pressure is measured and controlled over the blue tubing connected to the adapter over black angle connectors and routed to the two openings next to the camera. (C) The figure shows a rendering of the camera interface. The bottom fabric layer is sandwiched between the camera adapter and a 3D printed flange ring, which are fastened by six screws. The LED board and camera are inserted into the adapter from the top and the camera cable is routed through a slit in the top piece. Silicone glue is used to seal all interfaces.
• Figure 5: The picture shows a single actuator inflated to different expansions and the corresponding image from the internal camera. The number of cushion rings visible to the internal camera decreases as the actuator expands. The light intensity is set such that the white pattern is visible over the full range of the actuator expansion.