Privacy-Preserving in Connected and Autonomous Vehicles Through Vision to Text Transformation

TL;DR

This paper tackles privacy risks in ITS arising from visual data captured by AIE cameras in connected and autonomous vehicles. It introduces a privacy-preserving vision-to-text framework that converts images into descriptive text via a vision–language model, then refines the output through a hierarchical feedback-based reinforcement learning loop augmented by retrieval-augmented generation. The approach formalizes privacy via a differential privacy mechanism on sentence embeddings and combines PPO-based RL with external feedback to optimize semantic richness while minimizing private-content exposure, validated on CFP-FP and AgeDB-30 datasets. The results demonstrate improved privacy preservation alongside richer, semantically meaningful descriptions, enabling privacy-aware data sharing for smart city and industrial applications.

Abstract

Intelligent Transportation Systems (ITS) rely on a variety of devices that frequently process privacy-sensitive data. Roadside units are important because they use AI-equipped cameras to detect traffic violations in Connected and Autonomous Vehicles (CAV). However, although the interior of a vehicle is generally considered a private space, the privacy risks associated with captured imagery remain a major concern, as such data can be misused for identity theft, profiling, or unauthorized commercial purposes. Methods like face blurring reduce privacy risks, however individuals' privacy can still be compromised. This paper introduces a novel privacy-preserving framework that leverages feedback-based reinforcement learning (RL) and vision-language models (VLMs) to protect sensitive visual information captured by AIE cameras. The proposed idea transforms images into textual descriptions using an innovative method while the main scene details are preserved and protects privacy. A hierarchical RL strategy is employed to iteratively refine the generated text, enhancing both semantic accuracy and privacy. Unlike prior captioning-based methods, our model incorporates an iterative reinforcement-learning cycle with external knowledge feedback which progressively refines privacy-aware text. In addition to qualitative textual metric evaluations, the privacy-based metrics demonstrate significant improvements in privacy preservation where SSIM, PSNR, MSE, and SRRA values obtained using the proposed method on two different datasets outperform other methods.

Figures (9)

• Figure 1: Captured images are used to detect violations; however, as it is obvious, the privacy is compromised.
• Figure 2: (a) General models follow a straightforward process which captures an image performing the image processing task and then produces the output. (b) The proposed architecture begins with image capture and generates textual descriptions through an iterative cycle, using new prompts in each iteration.
• Figure 3: Feedback-based RL-VLM model running on AIE devices to capture images and transform into textual description through in an iterative process to generate maximum descriptions
• Figure 4: Hierarchical prompt selection where each layer prompts are selected at each iteration. As stated in Figure \ref{['RL_VLM']}, Step 3, the generated text is utilized to select the most relevant prompts from the prompt list during each iteration, where the list is updated in every cycle.
• Figure 5: The figure illustrates sample textual descriptions generated by the proposed model and two additional models. The first description is produced by a VLM-based model, while the second is generated by a VLM-RL model. The third description is produced by the proposed model, which is capable of generating significantly longer and more detailed textual outputs. Additionally, the red-colored highlights indicate textual richness, showing that the proposed model effectively captures the main elements of the image. Using the generated text, an text to image model was used to evaluate how close is the reconstructed image to the input image in terms of information leakage. It is obvious that the re-generated image is a vehicle but without any similarity to the input image, meanwhile safety.