Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Positive Experience Reflection for Agents in Interactive Text Environments

Philip Lippmann, Matthijs T. J. Spaan, Jie Yang

TL;DR

This work introduces Sweet&Sour, a novel approach that addresses limitations in existing reflection methods by incorporating positive experiences and managed memory to enrich the context available to the agent at decision time.

Abstract

Intelligent agents designed for interactive environments face significant challenges in text-based games, a domain that demands complex reasoning and adaptability. While agents based on large language models (LLMs) using self-reflection have shown promise, they struggle when initially successful and exhibit reduced effectiveness when using smaller LLMs. We introduce Sweet&Sour, a novel approach that addresses these limitations in existing reflection methods by incorporating positive experiences and managed memory to enrich the context available to the agent at decision time. Our comprehensive analysis spans both closed- and open-source LLMs and demonstrates the effectiveness of Sweet&Sour in improving agent performance, particularly in scenarios where previous approaches fall short.

Positive Experience Reflection for Agents in Interactive Text Environments

TL;DR

This work introduces Sweet&Sour, a novel approach that addresses limitations in existing reflection methods by incorporating positive experiences and managed memory to enrich the context available to the agent at decision time.

Abstract

Intelligent agents designed for interactive environments face significant challenges in text-based games, a domain that demands complex reasoning and adaptability. While agents based on large language models (LLMs) using self-reflection have shown promise, they struggle when initially successful and exhibit reduced effectiveness when using smaller LLMs. We introduce Sweet&Sour, a novel approach that addresses these limitations in existing reflection methods by incorporating positive experiences and managed memory to enrich the context available to the agent at decision time. Our comprehensive analysis spans both closed- and open-source LLMs and demonstrates the effectiveness of Sweet&Sour in improving agent performance, particularly in scenarios where previous approaches fall short.

Paper Structure

This paper contains 7 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Experiments
  4. Conclusion
  5. Background
  6. LLM Reflection
  7. Example ScienceWorld Task

Figures (1)

  • Figure 1: Comparison of used prompting methods to play ScienceWorld. ReAct introduces a THINK action to explicitly reason regarding the next step. Reflexion leverages self-reflection across attempts to learn from unsuccessful tries and stores these in memory. Sweet&Sour not only performs self-reflection after failures but also after each completed sub goal, making its reflection instantly available.