Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving agent performance in fluid environments by perceptual pretraining

Jin Zhang, Jianyang Xue, Bochao Cao

TL;DR

This work tackles cross-scenario agent performance in fluid environments by introducing a perceptual pretraining framework that compresses spatiotemporal flow information into a compact feature, $h_t$. The method combines a CNN-GRU perceptual network with contrastive predictive-like pretraining and PPO-based reinforcement learning to optimize drag reduction in a two-cylinder flow at $Re=100$. Key findings show that pretrained agents generalize better in obstacle-perception tasks and achieve significant drag reduction in RL tasks, accompanied by sensitivity analyses that reveal task-aligned attention patterns. The approach promises robust, multi-scenario capabilities for real-world fluid-embedded agents by leveraging unsupervised pretraining to shape transferable perceptual representations.

Abstract

In this paper, we construct a pretraining framework for fluid environment perception, which includes an information compression model and the corresponding pretraining method. We test this framework in a two-cylinder problem through numerical simulation. The results show that after unsupervised pretraining with this framework, the intelligent agent can acquire key features of surrounding fluid environment, thereby adapting more quickly and effectively to subsequent multi-scenario tasks. In our research, these tasks include perceiving the position of the upstream obstacle and actively avoiding shedding vortices in the flow field to achieve drag reduction. Better performance of the pretrained agent is discussed in the sensitivity analysis.

Improving agent performance in fluid environments by perceptual pretraining

TL;DR

This work tackles cross-scenario agent performance in fluid environments by introducing a perceptual pretraining framework that compresses spatiotemporal flow information into a compact feature, . The method combines a CNN-GRU perceptual network with contrastive predictive-like pretraining and PPO-based reinforcement learning to optimize drag reduction in a two-cylinder flow at . Key findings show that pretrained agents generalize better in obstacle-perception tasks and achieve significant drag reduction in RL tasks, accompanied by sensitivity analyses that reveal task-aligned attention patterns. The approach promises robust, multi-scenario capabilities for real-world fluid-embedded agents by leveraging unsupervised pretraining to shape transferable perceptual representations.

Abstract

In this paper, we construct a pretraining framework for fluid environment perception, which includes an information compression model and the corresponding pretraining method. We test this framework in a two-cylinder problem through numerical simulation. The results show that after unsupervised pretraining with this framework, the intelligent agent can acquire key features of surrounding fluid environment, thereby adapting more quickly and effectively to subsequent multi-scenario tasks. In our research, these tasks include perceiving the position of the upstream obstacle and actively avoiding shedding vortices in the flow field to achieve drag reduction. Better performance of the pretrained agent is discussed in the sensitivity analysis.
Paper Structure (12 sections, 3 equations, 10 figures, 3 tables)

This paper contains 12 sections, 3 equations, 10 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Numerical Method
  4. Perceptual network
  5. Reinforcement learning
  6. Results and discussion
  7. Pretraining
  8. Obstacle Perception
  9. Reinforcement Learning for drag reduction
  10. Sensitivity Analysis
  11. Conclusion
  12. Validation of the fluid solver

Figures (10)

  • Figure 1: Vorticity contours of the two-cylinder system. (a) Fixed obstacle cylinder; (b) Obstacle cylinder oscillating in the vertical direction.
  • Figure 2: Computational domain and grid setup. (a) Detailed view of the overset mesh around the cylinders. (b) Computational domain and boundary conditions.
  • Figure 3: Overview of the perceptual network architecture.
  • Figure 4: Loss curves for both pretrained and baseline agents.
  • Figure 5: Inferred versus real trajectories of the obstacle cylinder. (a) Test 1; (b) Test 2; (c) Test 3; (d) Test 4.
  • ...and 5 more figures