CaRoSaC: A Reinforcement Learning-Based Kinematic Control of Cable-Driven Parallel Robots by Addressing Cable Sag through Simulation

TL;DR

CaRoSaC tackles cable sag in CDPRs by pairing a sag-aware Unity3D-based simulator (CaRoSim) with a model-free TD3 controller that uses cable-length actions to achieve precise task-space positioning. A two-stage training pipeline (No-Sag followed by CaRoSim) and a carefully crafted reward design enable robust learning under complex sag dynamics, with superior performance over traditional kinematic solvers, particularly near workspace boundaries. Validation against real trajectories confirms high fidelity of CaRoSim and strong generalization of the learned policy to sag-related non-linearities. The work provides a reproducible, extensible framework for developing and evaluating sag-aware estimation and control strategies for suspended CDPRs.

Abstract

This paper introduces the Cable Robot Simulation and Control (CaRoSaC) Framework, which integrates a simulation environment with a model-free reinforcement learning control methodology for suspended Cable-Driven Parallel Robots (CDPRs), accounting for cable sag. Our approach seeks to bridge the knowledge gap of the intricacies of CDPRs due to aspects such as cable sag and precision control necessities by establishing a simulation platform that captures the real-world behaviors of CDPRs, including the impacts of cable sag. The framework offers researchers and developers a tool to further develop estimation and control strategies within the simulation for understanding and predicting the performance nuances, especially in complex operations where cable sag can be significant. Using this simulation framework, we train a model-free control policy in Reinforcement Learning (RL). This approach is chosen for its capability to adaptively learn from the complex dynamics of CDPRs. The policy is trained to discern optimal cable control inputs, ensuring precise end-effector positioning. Unlike traditional feedback-based control methods, our RL control policy focuses on kinematic control and addresses the cable sag issues without being tethered to predefined mathematical models. We also demonstrate that our RL-based controller, coupled with the flexible cable simulation, significantly outperforms the classical kinematics approach, particularly in dynamic conditions and near the boundary regions of the workspace. The combined strength of the described simulation and control approach offers an effective solution in manipulating suspended CDPRs even at workspace boundary conditions where traditional approach fails, as proven from our experiments, ensuring that CDPRs function optimally in various applications while accounting for the often neglected but critical factor of cable sag.

Figures (16)

• Figure 1: Reference CDPR system (Left) used for setting up our Unity3D simulation (Right) and for generating real-world trajectories.
• Figure 2: Kinematic diagram of our system, depicting all the parameters involved in formulating the kinematics and controller of suspended CDPR.
• Figure 3: Simulation architecture overview, showing different modules and their interactions. The core simulation script manages the individual sub-modules and provides the necessary information for the RL-agent controller.
• Figure 4: Overview of the architecture used for training a TD3 agent in a no-sag and CaRoSim environment.
• Figure 5: The behavior of all reward components described in Section \ref{['sec:methods']} is depicted. Each reward function is plotted against its respective dependent variable, shown on multiple x-axes with corresponding colors matching the reward curves. This visualization demonstrates how each reward depends on specific global information about the system.