RLInspect: An Interactive Visual Approach to Assess Reinforcement Learning Algorithm

TL;DR

RLInspect - an interactive visual analytic tool, that takes into account different components of the RL model - state, action, agent architecture and reward, and provides a more comprehensive view of the RL training.

Abstract

Reinforcement Learning (RL) is a rapidly growing area of machine learning that finds its application in a broad range of domains, from finance and healthcare to robotics and gaming. Compared to other machine learning techniques, RL agents learn from their own experiences using trial and error, and improve their performance over time. However, assessing RL models can be challenging, which makes it difficult to interpret their behaviour. While reward is a widely used metric to evaluate RL models, it may not always provide an accurate measure of training performance. In some cases, the reward may seem increasing while the model's performance is actually decreasing, leading to misleading conclusions about the effectiveness of the training. To overcome this limitation, we have developed RLInspect - an interactive visual analytic tool, that takes into account different components of the RL model - state, action, agent architecture and reward, and provides a more comprehensive view of the RL training. By using RLInspect, users can gain insights into the model's behaviour, identify issues during training, and potentially correct them effectively, leading to a more robust and reliable RL system.

Figures (8)

• Figure 1: Architecture of RLInspect
• Figure 2: Sequence of data flow in RLInspect
• Figure 3: State-space distribution in Cartpole.
• Figure 4: Exploration vs Exploitation in Cartpole.
• Figure 5: Training vs Non-Training states in Cartpole.

Theorems & Definitions (3)

• proof
• proof
• proof