Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Revelio: A Real-World Screen-Camera Communication System with Visually Imperceptible Data Embedding

Abbaas Alif Mohamed Nishar, Shrinivas Kudekar, Bernard Kintzing, Ashwin Ashok

TL;DR

Revelio tackles the challenge of embedding metadata in real-world video viewed on screens, aiming for imperceptibility and reliable smartphone-based decoding. It leverages spatially adaptive flicker in the OKLAB color space and encodes data into symbol-shaped regions, with a 288-bit RS-protected payload mapped across a 16x9 grid. Decoding employs a two-stage neural architecture for frame ROI detection and symbol recognition, combined with a weighted differential accumulator and time-diversity across multiple epochs, achieving robust performance up to about 2 meters under typical viewing conditions. The work demonstrates practical potential for interactive television and meta-information transmission, while outlining avenues to extend payload, range, and perceptual robustness.

Abstract

We present `Revelio', a real-world screen-camera communication system leveraging temporal flicker fusion in the OKLAB color space. Using spatially-adaptive flickering and encoding information in pixel region shapes, Revelio achieves visually imperceptible data embedding while remaining robust against noise, asynchronicity, and distortions in screen-camera channels, ensuring reliable decoding by standard smartphone cameras. The decoder, driven by a two-stage neural network, uses a weighted differential accumulator for precise frame detection and symbol recognition. Initial experiments demonstrate Revelio's effectiveness in interactive television, offering an unobtrusive method for meta-information transmission.

Revelio: A Real-World Screen-Camera Communication System with Visually Imperceptible Data Embedding

TL;DR

Revelio tackles the challenge of embedding metadata in real-world video viewed on screens, aiming for imperceptibility and reliable smartphone-based decoding. It leverages spatially adaptive flicker in the OKLAB color space and encodes data into symbol-shaped regions, with a 288-bit RS-protected payload mapped across a 16x9 grid. Decoding employs a two-stage neural architecture for frame ROI detection and symbol recognition, combined with a weighted differential accumulator and time-diversity across multiple epochs, achieving robust performance up to about 2 meters under typical viewing conditions. The work demonstrates practical potential for interactive television and meta-information transmission, while outlining avenues to extend payload, range, and perceptual robustness.

Abstract

We present `Revelio', a real-world screen-camera communication system leveraging temporal flicker fusion in the OKLAB color space. Using spatially-adaptive flickering and encoding information in pixel region shapes, Revelio achieves visually imperceptible data embedding while remaining robust against noise, asynchronicity, and distortions in screen-camera channels, ensuring reliable decoding by standard smartphone cameras. The decoder, driven by a two-stage neural network, uses a weighted differential accumulator for precise frame detection and symbol recognition. Initial experiments demonstrate Revelio's effectiveness in interactive television, offering an unobtrusive method for meta-information transmission.
Paper Structure (6 sections, 6 figures, 1 table)

This paper contains 6 sections, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Architecture
  3. Encoder
  4. Decoder
  5. Evaluation
  6. Conclusion

Figures (6)

  • Figure 1: Overall system: (i) A web app to upload videos and link meta-information to "Revelio codes"; (ii) a cloud-based encoder that invisibly embeds these codes; (iii) a mobile app, with cloud decoding, that extracts the hidden data by recording the screen.
  • Figure 2: Encoder pipeline: A 16-bit Revelio code is (Reed-Solomon) encoded to 288 bits, interleaved, mapped to a symbol, and flickered in the OKLAB space.
  • Figure 3: Best $(\lambda, \alpha, \beta)$ values for $B = 128$ and all $R, G$.
  • Figure 4: Decoder pipeline: A decoding epoch processes $N$ (typically 8 to 15) consecutive frames.
  • Figure 5: Error Rate vs. Distance
  • ...and 1 more figures