CURLing the Dream: Contrastive Representations for World Modeling in Reinforcement Learning

TL;DR

A novel reinforcement learning algorithm that integrates contrastive learning into the DreamerV3 framework to enhance performance in visual reinforcement learning tasks by incorporating the contrastive loss from the CURL algorithm and a reconstruction loss from autoencoder is presented.

Abstract

In this work, we present Curled-Dreamer, a novel reinforcement learning algorithm that integrates contrastive learning into the DreamerV3 framework to enhance performance in visual reinforcement learning tasks. By incorporating the contrastive loss from the CURL algorithm and a reconstruction loss from autoencoder, Curled-Dreamer achieves significant improvements in various DeepMind Control Suite tasks. Our extensive experiments demonstrate that Curled-Dreamer consistently outperforms state-of-the-art algorithms, achieving higher mean and median scores across a diverse set of tasks. The results indicate that the proposed approach not only accelerates learning but also enhances the robustness of the learned policies. This work highlights the potential of combining different learning paradigms to achieve superior performance in reinforcement learning applications.

Figures (3)

• Figure 1: Overview of the Curled-Dreamer algorithm, illustrating the integration of contrastive and reconstruction losses to enhance visual representation learning.
• Figure 2: Rendered demonstration for the following tasks: Acrobot Swingup, Finger Turn, Cheetah, and Quadruped.
• Figure 3: Evaluation returns on various DMC tasks for 1M steps.