Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review

TL;DR

This survey analyzes reinforcement learning as a framework for robotic control, tracing from fundamental MDP formalism to modern deep RL algorithms. It introduces a four-part taxonomy that clarifies how RL maps to robot competencies, learning pipelines, and deployment readiness. The paper reviews progress across locomotion, manipulation, navigation, and industrial automation, and critically assesses deployment challenges such as sample efficiency, sim-to-real gaps, and safety. It highlights future directions including human-in-the-loop, causal RL, and hybrid control to bridge theory and real-world robotic systems, underscoring the potential for RL to enable robust, autonomous robots in dynamic environments.

Abstract

Reinforcement learning (RL) has become a foundational approach for enabling intelligent robotic behavior in dynamic and uncertain environments. This work presents an in-depth review of RL principles, advanced deep reinforcement learning (DRL) algorithms, and their integration into robotic and control systems. Beginning with the formalism of Markov Decision Processes (MDPs), the study outlines essential elements of the agent-environment interaction and explores core algorithmic strategies including actor-critic methods, value-based learning, and policy gradients. Emphasis is placed on modern DRL techniques such as DDPG, TD3, PPO, and SAC, which have shown promise in solving high-dimensional, continuous control tasks. A structured taxonomy is introduced to categorize RL applications across domains such as locomotion, manipulation, multi-agent coordination, and human-robot interaction, along with training methodologies and deployment readiness levels. The review synthesizes recent research efforts, highlighting technical trends, design patterns, and the growing maturity of RL in real-world robotics. Overall, this work aims to bridge theoretical advances with practical implementations, providing a consolidated perspective on the evolving role of RL in autonomous robotic systems.

Figures (4)

• Figure 1: Illustration of the Actor-Critic reinforcement learning framework. The agent receives a state $s_t$ from the environment, selects an action $a_t$ via a policy $\pi(s)$, and receives a reward $r_t$. The critic evaluates the action using a value function $V^\pi(s)$.
• Figure 2: Taxonomy of RL Applications in Robotics and Control Systems. Components (a)--(d) depict robot task categories, training structure, learning pipeline, and deployment readiness levels tang2025deep
• Figure 3: Review of Reinforcement Learning Applications in Robotics and Control Systems
• Figure 4: Review of Reinforcement Learning Applications in Robotics and Control Systems