Open-Source LLM-Driven Federated Transformer for Predictive IoV Management

TL;DR

Open-Source LLM-Driven Federated Transformer for Predictive IoV Management proposes FPoTT, a privacy-preserving IoV framework that unites dynamic prompt optimization, dual-layer cloud–edge federated learning, and Transformer-based synthetic data generation. The approach enables real-time trajectory prediction using open-source LLMs, notably EleutherAI Pythia-1B, achieving around $99.86\%$ accuracy on real NGSIM data and maintaining strong performance on synthetic data. Key contributions include a Prompt Generator for adaptive textual prompts, a two-tier federated architecture for low-latency inference with global knowledge sharing, and a Transformer-based NGSIM data generator to enrich training. The work demonstrates the viability of open-source LLMs for secure, scalable IoV management, offering a cost-effective alternative to proprietary models and highlighting avenues for stronger privacy guarantees and broader deployment in smart mobility ecosystems.

Abstract

The proliferation of connected vehicles within the Internet of Vehicles (IoV) ecosystem presents critical challenges in ensuring scalable, real-time, and privacy-preserving traffic management. Existing centralized IoV solutions often suffer from high latency, limited scalability, and reliance on proprietary Artificial Intelligence (AI) models, creating significant barriers to widespread deployment, particularly in dynamic and privacy-sensitive environments. Meanwhile, integrating Large Language Models (LLMs) in vehicular systems remains underexplored, especially concerning prompt optimization and effective utilization in federated contexts. To address these challenges, we propose the Federated Prompt-Optimized Traffic Transformer (FPoTT), a novel framework that leverages open-source LLMs for predictive IoV management. FPoTT introduces a dynamic prompt optimization mechanism that iteratively refines textual prompts to enhance trajectory prediction. The architecture employs a dual-layer federated learning paradigm, combining lightweight edge models for real-time inference with cloud-based LLMs to retain global intelligence. A Transformer-driven synthetic data generator is incorporated to augment training with diverse, high-fidelity traffic scenarios in the Next Generation Simulation (NGSIM) format. Extensive evaluations demonstrate that FPoTT, utilizing EleutherAI Pythia-1B, achieves 99.86% prediction accuracy on real-world data while maintaining high performance on synthetic datasets. These results underscore the potential of open-source LLMs in enabling secure, adaptive, and scalable IoV management, offering a promising alternative to proprietary solutions in smart mobility ecosystems.

Figures (5)

• Figure 1: Simplified Overview of the proposed framework. The system consists of three components: a prompt optimizer that refines input prompts based on evaluation metrics, a training pipeline using both real and synthetic NGSIM data to fine-tune LLMs, and an inference stage for real-time traffic prediction.
• Figure 2: Illustration of FPoTT's federated learning architecture where a central cloud server distributes and aggregates model parameters $\theta_c$, $\theta_e$ to and from edge clients running local small LLMs on private local data. In real-world applications, several clients could be running under the same server.
• Figure 3: Accuracy comparison of different models using default prompts on NGSIM and simulated datasets. EleutherAI/pythia-1b, GPT-2-Medium (355M), and OPT-1.3b are evaluated to assess baseline model performance in IoV trajectory prediction tasks.
• Figure 4: Accuracy comparison of different models under low complexity prompts using NGSIM and simulated datasets. EleutherAI/pythia-1b, GPT-2-Medium (355M), and OPT-1.3b are evaluated to assess model performance in privacy-preserving IoV scenarios.
• Figure 5: Accuracy comparison of different models under high complexity prompts using NGSIM and simulated datasets. EleutherAI/pythia-1b, GPT-2-Medium (355M), and OPT-1.3b are evaluated to assess the effect of prompt refinement on trajectory prediction in IoV management.