Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Obfuscation Based Privacy Preserving Representations are Recoverable Using Neighborhood Information

Kunal Chelani, Assia Benbihi, Fredrik Kahl, Torsten Sattler, Zuzana Kukelova

TL;DR

This work shows that geometry-based privacy-preserving obfuscations used in visual localization are vulnerable when neighborhood information is available, enabling recovery of original point positions and image inversion across common schemes. It introduces a simple optimization-based recovery framework that leverages obfuscated neighborhoods, and a learning-based method to infer neighborhoods from descriptors for end-to-end attacks. Extensive experiments across multiple 2D/3D obfuscations and datasets demonstrate that private content can be recovered, challenging the claimed privacy guarantees. The findings emphasize the need for combined descriptor- and geometry-level defenses and formal privacy guarantees in localization systems.

Abstract

Rapid growth in the popularity of AR/VR/MR applications and cloud-based visual localization systems has given rise to an increased focus on the privacy of user content in the localization process. This privacy concern has been further escalated by the ability of deep neural networks to recover detailed images of a scene from a sparse set of 3D or 2D points and their descriptors - the so-called inversion attacks. Research on privacy-preserving localization has therefore focused on preventing these inversion attacks on both the query image keypoints and the 3D points of the scene map. To this end, several geometry obfuscation techniques that lift points to higher-dimensional spaces, i.e., lines or planes, or that swap coordinates between points % have been proposed. In this paper, we point to a common weakness of these obfuscations that allows to recover approximations of the original point positions under the assumption of known neighborhoods. We further show that these neighborhoods can be computed by learning to identify descriptors that co-occur in neighborhoods. Extensive experiments show that our approach for point recovery is practically applicable to all existing geometric obfuscation schemes. Our results show that these schemes should not be considered privacy-preserving, even though they are claimed to be privacy-preserving. Code will be available at https://github.com/kunalchelani/RecoverPointsNeighborhood.

Obfuscation Based Privacy Preserving Representations are Recoverable Using Neighborhood Information

TL;DR

This work shows that geometry-based privacy-preserving obfuscations used in visual localization are vulnerable when neighborhood information is available, enabling recovery of original point positions and image inversion across common schemes. It introduces a simple optimization-based recovery framework that leverages obfuscated neighborhoods, and a learning-based method to infer neighborhoods from descriptors for end-to-end attacks. Extensive experiments across multiple 2D/3D obfuscations and datasets demonstrate that private content can be recovered, challenging the claimed privacy guarantees. The findings emphasize the need for combined descriptor- and geometry-level defenses and formal privacy guarantees in localization systems.

Abstract

Rapid growth in the popularity of AR/VR/MR applications and cloud-based visual localization systems has given rise to an increased focus on the privacy of user content in the localization process. This privacy concern has been further escalated by the ability of deep neural networks to recover detailed images of a scene from a sparse set of 3D or 2D points and their descriptors - the so-called inversion attacks. Research on privacy-preserving localization has therefore focused on preventing these inversion attacks on both the query image keypoints and the 3D points of the scene map. To this end, several geometry obfuscation techniques that lift points to higher-dimensional spaces, i.e., lines or planes, or that swap coordinates between points % have been proposed. In this paper, we point to a common weakness of these obfuscations that allows to recover approximations of the original point positions under the assumption of known neighborhoods. We further show that these neighborhoods can be computed by learning to identify descriptors that co-occur in neighborhoods. Extensive experiments show that our approach for point recovery is practically applicable to all existing geometric obfuscation schemes. Our results show that these schemes should not be considered privacy-preserving, even though they are claimed to be privacy-preserving. Code will be available at https://github.com/kunalchelani/RecoverPointsNeighborhood.
Paper Structure (11 sections, 4 equations, 18 figures, 15 tables)

This paper contains 11 sections, 4 equations, 18 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Geometric Obfuscation of Points
  4. Recovering Obfuscated Points using Neighborhood Information
  5. Estimating Neighborhoods From Descriptors
  6. Experimental Evaluation
  7. Conclusion
  8. Coordinate permutation - Predicting swapped coordinate
  9. Descriptor Assignment for Paired-Point-Lines
  10. Additional Results
  11. Implementation Details

Figures (18)

  • Figure 1: Geometry obfuscations allow the recovery of image details. The original point representations are privacy revealing as full images can be recovered from them pittaluga2019revealing. Different obfuscation schemes are used to modify them. In this paper, we show that given neighborhood information, it is possible to approximately recover the original point positions, again enabling image recovery.
  • Figure 2: Visual content revealed by the inversion pittaluga2019revealing from the original points ('Baseline') and the points recovered from the 3D obfuscations with neighborhood information at various levels of inlier ratios (In.). The original points are triangulated from SIFT Lowe04IJCV features. Line obfuscations (OLC) speciale2019privacyspeciale2019privacy2d, Point-Pair-Lines PPL lee2023paired and RayClouds moon2024efficient are more vulnerable to neighborhood-based attacks than Planes geppert2022privacy and Coordinate Permutation pan2023privacy.
  • Figure 3: (Best viewed when zoomed in.) Visual content revealed by the inversion applied on points recovered from the obfuscated representations when using two different kinds of keypoints extractors and descriptors - SuperPoint detone2018superpoint and SIFT Lowe04IJCV. The columns titled Estimated NN show the content revealed with an end-to-end attack, i.e., starting from only descriptors, we carry out neighborhood estimation, point recovery, and inversion to the image space. The presence of identifiable scene content in the inverted images emphasizes the vulnerability of current geometry obfuscation techniques.
  • Figure 4: Illustration of the Coordinate Swap Inversion. The green points represent the true original points that form a neighborhood. One coordinate of each point is swapped with that of another point in the image (not shown here for brevity) to result in the blue/pink points. Note that points shifted along the y-axis (pink) form a cluster around the same x-value and similarly points shifted along the x-axis form a cluster around the same y-value. This idea is used to estimate the swapped coordinates of the members of a neighborhood.
  • Figure 5: Additional Qualitative Results - 7-scenes shotton2013scene-Chess. Images inverted pittaluga2019revealing from the original points ('Baseline') and the points recovered from the 3D obfuscations from neighborhood information with various levels of inlier ratios (In.). Line obfuscations (OLC) speciale2019privacyspeciale2019privacy2d, Point-Pair-Lines PPL and PPL+ lee2023paired, and ray clouds moon2024efficient are the most vulnerable to neighborhood-based attacks while Planes geppert2022privacy and Permutations pan2023privacy are more privacy preserving. The 3D points cloud is generated from SfM schonberger2016structure on SIFT Lowe04IJCV features.
  • ...and 13 more figures