Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pose-Based Sign Language Appearance Transfer

Amit Moryossef, Gerard Sant, Zifan Jiang

TL;DR

This paper addresses privacy concerns in sign language data by introducing a pose-based appearance transfer that preserves sign content while altering signer appearance. The method normalizes poses and replaces signer appearance using a mean or alternate appearance, enabling smoother rendering and stitching without modifying hand geometry. Experiments on AUTSL show a clear drop in signer-identification accuracy, indicating improved privacy, but a slight decline in sign recognition, revealing a privacy–utility tradeoff. The work highlights practical implications for privacy-preserving sign language processing and outlines future directions, including segmentation-based anonymization and transcription to ensure robust anonymity, with code available online.

Abstract

We introduce a method for transferring the signer's appearance in sign language skeletal poses while preserving the sign content. Using estimated poses, we transfer the appearance of one signer to another, maintaining natural movements and transitions. This approach improves pose-based rendering and sign stitching while obfuscating identity. Our experiments show that while the method reduces signer identification accuracy, it slightly harms sign recognition performance, highlighting a tradeoff between privacy and utility. Our code is available at https://github.com/sign-language-processing/pose-anonymization.

Pose-Based Sign Language Appearance Transfer

TL;DR

This paper addresses privacy concerns in sign language data by introducing a pose-based appearance transfer that preserves sign content while altering signer appearance. The method normalizes poses and replaces signer appearance using a mean or alternate appearance, enabling smoother rendering and stitching without modifying hand geometry. Experiments on AUTSL show a clear drop in signer-identification accuracy, indicating improved privacy, but a slight decline in sign recognition, revealing a privacy–utility tradeoff. The work highlights practical implications for privacy-preserving sign language processing and outlines future directions, including segmentation-based anonymization and transcription to ensure robust anonymity, with code available online.

Abstract

We introduce a method for transferring the signer's appearance in sign language skeletal poses while preserving the sign content. Using estimated poses, we transfer the appearance of one signer to another, maintaining natural movements and transitions. This approach improves pose-based rendering and sign stitching while obfuscating identity. Our experiments show that while the method reduces signer identification accuracy, it slightly harms sign recognition performance, highlighting a tradeoff between privacy and utility. Our code is available at https://github.com/sign-language-processing/pose-anonymization.

Paper Structure

This paper contains 8 sections, 1 equation, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Qualitative Evaluation
  5. Rendering
  6. Sign Stitching
  7. Experiments and Results
  8. Conclusions

Figures (3)

  • Figure 1: The average MediaPipe Holistic frame (landmarks reduced for visual clarity) extracted from a large sign language dataset ($\approx50$ million frames).
  • Figure 2: Faces from ControlNet Rendering
  • Figure 3: Optical flow (the magnitude of change between two frames) for a stitched video from four original videos and anonymized videos. Higher values represent a larger local change, and a higher area under the curve represents a larger change overall. The flow is exactly the same for all frames except for the stitching zones.