Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SVIA: A Street View Image Anonymization Framework for Self-Driving Applications

Dongyu Liu, Xuhong Wang, Cen Chen, Yanhao Wang, Shengyue Yao, Yilun Lin

TL;DR

This work tackles privacy leakage in street-view imagery used for self-driving by extending anonymization beyond faces to street-level content. It introduces SVIA, a three-component pipeline that combines semantic segmentation, latent-diffusion–based inpainting, and a harmonizer to produce high-quality, privacy-preserving street-view images. Experimental results on Cityscapes and Mapillary Vistas show SVIA achieves a superior balance between image realism and privacy protection compared with baselines, while maintaining downstream task utility. The method has practical potential for real-world deployment in autonomous driving data pipelines, though it currently lacks video support and operates relatively slowly, suggesting avenues for future acceleration and extension to street-view videos.

Abstract

In recent years, there has been an increasing interest in image anonymization, particularly focusing on the de-identification of faces and individuals. However, for self-driving applications, merely de-identifying faces and individuals might not provide sufficient privacy protection since street views like vehicles and buildings can still disclose locations, trajectories, and other sensitive information. Therefore, it remains crucial to extend anonymization techniques to street view images to fully preserve the privacy of users, pedestrians, and vehicles. In this paper, we propose a Street View Image Anonymization (SVIA) framework for self-driving applications. The SVIA framework consists of three integral components: a semantic segmenter to segment an input image into functional regions, an inpainter to generate alternatives to privacy-sensitive regions, and a harmonizer to seamlessly stitch modified regions to guarantee visual coherence. Compared to existing methods, SVIA achieves a much better trade-off between image generation quality and privacy protection, as evidenced by experimental results for five common metrics on two widely used public datasets.

SVIA: A Street View Image Anonymization Framework for Self-Driving Applications

TL;DR

This work tackles privacy leakage in street-view imagery used for self-driving by extending anonymization beyond faces to street-level content. It introduces SVIA, a three-component pipeline that combines semantic segmentation, latent-diffusion–based inpainting, and a harmonizer to produce high-quality, privacy-preserving street-view images. Experimental results on Cityscapes and Mapillary Vistas show SVIA achieves a superior balance between image realism and privacy protection compared with baselines, while maintaining downstream task utility. The method has practical potential for real-world deployment in autonomous driving data pipelines, though it currently lacks video support and operates relatively slowly, suggesting avenues for future acceleration and extension to street-view videos.

Abstract

In recent years, there has been an increasing interest in image anonymization, particularly focusing on the de-identification of faces and individuals. However, for self-driving applications, merely de-identifying faces and individuals might not provide sufficient privacy protection since street views like vehicles and buildings can still disclose locations, trajectories, and other sensitive information. Therefore, it remains crucial to extend anonymization techniques to street view images to fully preserve the privacy of users, pedestrians, and vehicles. In this paper, we propose a Street View Image Anonymization (SVIA) framework for self-driving applications. The SVIA framework consists of three integral components: a semantic segmenter to segment an input image into functional regions, an inpainter to generate alternatives to privacy-sensitive regions, and a harmonizer to seamlessly stitch modified regions to guarantee visual coherence. Compared to existing methods, SVIA achieves a much better trade-off between image generation quality and privacy protection, as evidenced by experimental results for five common metrics on two widely used public datasets.
Paper Structure (20 sections, 6 figures, 3 tables, 3 algorithms)

This paper contains 20 sections, 6 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Anonymization
  4. Semantic Segmentation
  5. Diffusion-based Image Generation
  6. Image Inpainting
  7. Our Framework
  8. Background on Street View Image Anonymization
  9. Overview of SVIA Framework
  10. Semantic Segmentation
  11. Image Inpainting
  12. Harmonizer
  13. Experiments
  14. Setup
  15. Evaluation Metrics
  16. ...and 5 more sections

Figures (6)

  • Figure 1: Illustration of Grad-CAM results for city classification, where the regions with higher scores for city re-identification are indicated in red and yellow colors. This reveals that roads and buildings are the primary semantic categories to identify the city where a street view image is taken.
  • Figure 2: Overview of the SVIA pipeline.
  • Figure 3: Detailed Architectures of (A) the segmenter, (B) the inpainter, and (C) the harmonizer.
  • Figure 4: Illustration of the trade-off between image quality (KID) and privacy protection (LPIPS) for each method. The closer the point w.r.t. a method is to the upper right corner, the more favorable its overall performance is.
  • Figure 5: Visual comparison of four original images and their corresponding images generated by SVIA w/o Harmonizer, SVIA, and MAT+Harmonizer.
  • ...and 1 more figures