Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accurate Open-Loop Control of a Soft Continuum Robot Through Visually Learned Latent Representations

Henrik Krauss, Johann Licher, Naoya Takeishi, Annika Raatz, Takehisa Yairi

Abstract

This work addresses open-loop control of a soft continuum robot (SCR) from video-learned latent dynamics. Visual Oscillator Networks (VONs) from previous work are used, that provide mechanistically interpretable 2D oscillator latents through an attention broadcast decoder (ABCD). Open-loop, single-shooting optimal control is performed in latent space to track image-specified waypoints without camera feedback. An interactive SCR live simulator enables design of static, dynamic, and extrapolated targets and maps them to model-specific latent waypoints. On a two-segment pneumatic SCR, Koopman, MLP, and oscillator dynamics, each with and without ABCD, are evaluated on setpoint and dynamic trajectories. ABCD-based models consistently reduce image-space tracking error. The VON and ABCD-based Koopman models attains the lowest MSEs. Using an ablation study, we demonstrate that several architecture choices and training settings contribute to the open-loop control performance. Simulation stress tests further confirm static holding, stable extrapolated equilibria, and plausible relaxation to the rest state. To the best of our knowledge, this is the first demonstration that interpretable, video-learned latent dynamics enable reliable long-horizon open-loop control of an SCR.

Accurate Open-Loop Control of a Soft Continuum Robot Through Visually Learned Latent Representations

Abstract

This work addresses open-loop control of a soft continuum robot (SCR) from video-learned latent dynamics. Visual Oscillator Networks (VONs) from previous work are used, that provide mechanistically interpretable 2D oscillator latents through an attention broadcast decoder (ABCD). Open-loop, single-shooting optimal control is performed in latent space to track image-specified waypoints without camera feedback. An interactive SCR live simulator enables design of static, dynamic, and extrapolated targets and maps them to model-specific latent waypoints. On a two-segment pneumatic SCR, Koopman, MLP, and oscillator dynamics, each with and without ABCD, are evaluated on setpoint and dynamic trajectories. ABCD-based models consistently reduce image-space tracking error. The VON and ABCD-based Koopman models attains the lowest MSEs. Using an ablation study, we demonstrate that several architecture choices and training settings contribute to the open-loop control performance. Simulation stress tests further confirm static holding, stable extrapolated equilibria, and plausible relaxation to the rest state. To the best of our knowledge, this is the first demonstration that interpretable, video-learned latent dynamics enable reliable long-horizon open-loop control of an SCR.
Paper Structure (11 sections, 16 equations, 4 figures, 1 table)

This paper contains 11 sections, 16 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Learning Latent Dynamics from Video
  4. Predictive Control in Latent Space
  5. Obtaining Target Observations Through an SCR Live Simulator
  6. Experiments
  7. Hardware, Dataset, and Training of Models
  8. Open-Loop Control
  9. Ablation study
  10. Simulational experiments
  11. Conclusions

Figures (4)

  • Figure 1: Overview of this study's approach: an SCR live simulator is employed to generate target states for open-loop optimal control of the physical SCR. The feasibility of this method is validated across diverse trajectories.
  • Figure 2: (a) Model performances in open-loop control over various trajectory types (image MSEs). (b) MAE of input pressure prediction of open-loop setpoint control over multi-step prediction errors (50 samples each), including ablations.
  • Figure 3: (a) Examples for trajectories evaluated for open-loop optimal open-loop control. The real trajectories shown are based on the VON model. All target trajectories are accurately achieved, except the upswing, where an additional plot highlights a deviation between set and measured pressure inputs. (b) Predicted and real latent trajectories and input pressures for the fast dynamic trajectory and upswing trajectory (c), depicted on the left.
  • Figure 4: Simulation results for three stress tests. (a) Static holding at constant pressure for 50 static states. (b) Cosine ramp-up to extrapolated pressures. (c) Cosine excitation within dataset limits followed by a release. The upper row shows image-space MSEs over time. Below are expected final observation in (a) and (c), and (b) visualizes VON oscillator and stiffness forces. The lower-right of each column shows decoded observations at the final simulation step. ABCD-based models provide more stable and reasonable predictions.