CLSP: High-Fidelity Contrastive Language-State Pre-training for Agent State Representation

TL;DR

CLSP tackles the challenge of encoding scalar-based agent states into high-fidelity representations for RL and multimodal LLMs. It combines a coarse-grained classification pre-training stage with a CLIP-style contrastive language–state alignment, and augments scalar encoding with Random Fourier Features to preserve numerical precision, formalized through $\gamma(v)=[\cos(2\pi \mathbf{b} v), \sin(2\pi \mathbf{b} v)]^{\mathrm{T}}$. Empirical results on a FPS dataset with ~0.55M state-text pairs show improvements in text–state retrieval, faster RL navigation with higher GCR, and superior multimodal LLM understanding, supported by ablations and visualization. Overall, CLSP provides a practical, generalizable framework for high-fidelity state representation in embodied AI and multimodal systems.

Abstract

With the rapid development of artificial intelligence, multimodal learning has become an important research area. For intelligent agents, the state is a crucial modality to convey precise information alongside common modalities like images, videos, and language. This becomes especially clear with the broad adoption of reinforcement learning and multimodal large language models. Nevertheless, the representation of state modality still lags in development. To this end, we propose a High-Fidelity Contrastive Language-State Pre-training (CLSP) method, which can accurately encode state information into general representations for both reinforcement learning and multimodal large language models. Specifically, we first design a pre-training task based on the classification to train an encoder with coarse-grained information. Next, we construct data pairs of states and language descriptions, utilizing the pre-trained encoder to initialize the CLSP encoder. Then, we deploy contrastive learning to train the CLSP encoder to effectively represent precise state information. Additionally, we enhance the representation of numerical information using the Random Fourier Features (RFF) method for high-fidelity mapping. Extensive experiments demonstrate the superior precision and generalization capabilities of our representation, achieving outstanding results in text-state retrieval, reinforcement learning navigation tasks, and multimodal large language model understanding.

Figures (5)

• Figure 1: The overall structure of CLSP has three steps. The first step is classification-based pre-training, which aims to obtain coarse-grained information. The second step is to align states and the corresponding text descriptions. The third step is to apply the learned CLSP to subsequent tasks.
• Figure 2: The architecture of multimodal LLM with our state encoder, and the example of state-based QA.
• Figure 3: Illustration of Top-1 MAE curves during the training procedure for different methods.
• Figure 4: Illustration of goal completion ratio during the training procedure for the RL navigation task.
• Figure 5: The t-SNE visualization of state representations produced by CLSP.