Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ConvXformer: Differentially Private Hybrid ConvNeXt-Transformer for Inertial Navigation

Omer Tariq, Muhammad Bilal, Muneeb Ul Hassan, Dongsoo Han, Jon Crowcroft

TL;DR

ConvXformer addresses privacy in data-driven inertial navigation by co-designing a ConvNeXt-Transformer hybrid with a novel gradient-aligned DP mechanism. The method uses adaptive gradient clipping and truncated SVD with GANI to protect sensitive training data while preserving trajectory accuracy. Evaluations on RIDI, RoNIN, OxIOD, and the new Mech-IO dataset show substantial improvements in positioning accuracy under $(\epsilon,\delta)$-DP with competitive privacy guarantees. The work introduces Mech-IO to test robustness under magnetic interference, underscoring practical applicability in cyber-physical security for indoor/outdoor navigation.

Abstract

Data-driven inertial sequence learning has revolutionized navigation in GPS-denied environments, offering superior odometric resolution compared to traditional Bayesian methods. However, deep learning-based inertial tracking systems remain vulnerable to privacy breaches that can expose sensitive training data. \hl{Existing differential privacy solutions often compromise model performance by introducing excessive noise, particularly in high-frequency inertial measurements.} In this article, we propose ConvXformer, a hybrid architecture that fuses ConvNeXt blocks with Transformer encoders in a hierarchical structure for robust inertial navigation. We propose an efficient differential privacy mechanism incorporating adaptive gradient clipping and gradient-aligned noise injection (GANI) to protect sensitive information while ensuring model performance. Our framework leverages truncated singular value decomposition for gradient processing, enabling precise control over the privacy-utility trade-off. Comprehensive performance evaluations on benchmark datasets (OxIOD, RIDI, RoNIN) demonstrate that ConvXformer surpasses state-of-the-art methods, achieving more than 40% improvement in positioning accuracy while ensuring $(ε,δ)$-differential privacy guarantees. To validate real-world performance, we introduce the Mech-IO dataset, collected from the mechanical engineering building at KAIST, where intense magnetic fields from industrial equipment induce significant sensor perturbations. This demonstrated robustness under severe environmental distortions makes our framework well-suited for secure and intelligent navigation in cyber-physical systems.

ConvXformer: Differentially Private Hybrid ConvNeXt-Transformer for Inertial Navigation

TL;DR

ConvXformer addresses privacy in data-driven inertial navigation by co-designing a ConvNeXt-Transformer hybrid with a novel gradient-aligned DP mechanism. The method uses adaptive gradient clipping and truncated SVD with GANI to protect sensitive training data while preserving trajectory accuracy. Evaluations on RIDI, RoNIN, OxIOD, and the new Mech-IO dataset show substantial improvements in positioning accuracy under -DP with competitive privacy guarantees. The work introduces Mech-IO to test robustness under magnetic interference, underscoring practical applicability in cyber-physical security for indoor/outdoor navigation.

Abstract

Data-driven inertial sequence learning has revolutionized navigation in GPS-denied environments, offering superior odometric resolution compared to traditional Bayesian methods. However, deep learning-based inertial tracking systems remain vulnerable to privacy breaches that can expose sensitive training data. \hl{Existing differential privacy solutions often compromise model performance by introducing excessive noise, particularly in high-frequency inertial measurements.} In this article, we propose ConvXformer, a hybrid architecture that fuses ConvNeXt blocks with Transformer encoders in a hierarchical structure for robust inertial navigation. We propose an efficient differential privacy mechanism incorporating adaptive gradient clipping and gradient-aligned noise injection (GANI) to protect sensitive information while ensuring model performance. Our framework leverages truncated singular value decomposition for gradient processing, enabling precise control over the privacy-utility trade-off. Comprehensive performance evaluations on benchmark datasets (OxIOD, RIDI, RoNIN) demonstrate that ConvXformer surpasses state-of-the-art methods, achieving more than 40% improvement in positioning accuracy while ensuring -differential privacy guarantees. To validate real-world performance, we introduce the Mech-IO dataset, collected from the mechanical engineering building at KAIST, where intense magnetic fields from industrial equipment induce significant sensor perturbations. This demonstrated robustness under severe environmental distortions makes our framework well-suited for secure and intelligent navigation in cyber-physical systems.
Paper Structure (39 sections, 5 theorems, 50 equations, 9 figures, 3 tables, 1 algorithm)

This paper contains 39 sections, 5 theorems, 50 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Deep Learning-based Inertial Navigation Frameworks
  4. Differential Privacy for Location-based services
  5. Preliminaries
  6. Data-driven Inertial Navigation Approach
  7. Differential Privacy: Definitions and Mechanisms
  8. Differentially Private Stochastic Gradient Descent (DP-SGD)
  9. Methodology
  10. Proposed Data Augmentation Technique
  11. Model Architecture and Design
  12. Overall Architecture
  13. Stage Configuration
  14. ConvNeXt Block with Integrated Transformer Encoder
  15. Micro Design
  16. ...and 24 more sections

Key Result

Theorem 1

For the truncated SVD mechanism with adaptive clipping threshold $\lambda_t$, rank $k$, and adjacent datasets $\mathcal{D}$ and $\mathcal{D}'$, the $\ell_2$-sensitivity satisfies: where $\nabla\theta_k$ and $\nabla\theta'_k$ are the truncated gradient tensors computed on $\mathcal{D}$ and $\mathcal{D}'$ respectively.

Figures (9)

  • Figure 1: World and IMU coordinate frames: (a) ECEF frame {e} at Earth's center, with x-axis toward the prime meridian and equator, z-axis toward the North Pole; (b) ENU {n} and local {l} frames on the ground, z-axis upward; (c) IMU frame {imu} attached to the sensor.
  • Figure 2: Differentially Private Training Framework for ConvXformer.
  • Figure 3: Overall architecture of ConvXformer. The design strategy is based on a pyramid structure with four stages similar to convnext. Each stage comprises two ConvNeXt blocks with integrated Transformer encoders hierarchically. This figure shows the configuration of the proposed ConvXformer where $\{B_1, B_2, B_3, B_4\}$={2,2,2,2}.
  • Figure 4: Overview of the proposed differential privacy mechanism. Adaptive gradient clipping dynamically adjusts thresholds based on historical norms, followed by gradient-aligned noise injection (GANI), which introduces utility-weighted Gaussian noise aligned with principal gradient directions.
  • Figure 5: Smartphone IMU data acquisition, three participants carried their smartphones in five positions: walking, talking, bag, inside bag, and pocket.
  • ...and 4 more figures

Theorems & Definitions (11)

  • Definition 1
  • Theorem 1: SVD Sensitivity
  • proof
  • Lemma 1: Gradient-Aligned Noise Composition
  • proof
  • Theorem 2: Overall Privacy Guarantee
  • proof
  • Lemma 2: Privacy Amplification
  • proof
  • Lemma 3: Convergence Analysis
  • ...and 1 more