BLIP-FusePPO: A Vision-Language Deep Reinforcement Learning Framework for Lane Keeping in Autonomous Vehicles

TL;DR

BLIP-FusePPO presents a multimodal reinforcement learning framework for autonomous lane keeping that directly injects vision-language semantic embeddings from BLIP into the agent's state alongside geometric, LiDAR, and PID signals. By fusing these modalities and employing a hybrid reward that combines semantic alignment with geometric cues, the method achieves improved stability, convergence, and lateral accuracy compared to vision-based and multimodal baselines. The approach includes a PID-driven state augmentation, data augmentation, and a LoRA-enhanced BLIP processing workflow to maintain efficiency. Results in a Webots-based simulation show significant gains in LK performance, with BLIP-FusePPO achieving lower RMSE and tighter steering distributions, indicating strong potential for real-time deployment in diverse driving conditions.

Abstract

In this paper, we propose Bootstrapped Language-Image Pretraining-driven Fused State Representation in Proximal Policy Optimization (BLIP-FusePPO), a novel multimodal reinforcement learning (RL) framework for autonomous lane-keeping (LK), in which semantic embeddings generated by a vision-language model (VLM) are directly fused with geometric states, LiDAR observations, and Proportional-Integral-Derivative-based (PID) control feedback within the agent observation space. The proposed method lets the agent learn driving rules that are aware of their surroundings and easy to understand by combining high-level scene understanding from the VLM with low-level control and spatial signals. Our architecture brings together semantic, geometric, and control-aware representations to make policy learning more robust. A hybrid reward function that includes semantic alignment, LK accuracy, obstacle avoidance, and speed regulation helps learning to be more efficient and generalizable. Our method is different from the approaches that only use semantic models to shape rewards. Instead, it directly embeds semantic features into the state representation. This cuts down on expensive runtime inference and makes sure that semantic guidance is always available. The simulation results show that the proposed model is better at LK stability and adaptability than the best vision-based and multimodal RL baselines in a wide range of difficult driving situations. We make our code publicly available.

Figures (8)

• Figure 1: A brief look at the BLIP-FusePPO pipeline. The agent receives data from cameras, LiDAR, PID error feedback, and semantic embeddings from a BLIP. These are processed and combined into a hybrid state vector, which is then passed to a policy trained using PPO.
• Figure 2: Pipeline for semantic token generation via VLM. Captions are generated and tokenized into fixed-length embeddings.
• Figure 3: Symmetric augmentation improves generalization and helps mitigate directional bias in training data.
• Figure 4: PPO update. The actor and critic networks share a fused state vector composed of visual, LiDAR, PID, and semantic (BLIP) embeddings.
• Figure 5: Steering angle distributions for various agent configurations. The PID+VLM-enhanced agent exhibits tightly clustered, low-variance steering angles near zero (straight driving), reflecting more stable policy behavior.