Transformers -- Messages in Disguise

TL;DR

The Random Adversarial Data Obfuscation Model (RANDOM) Adversarial Neural Cryptography (ANC) network is introduced, which results in an ANC network that is computationally efficient, ensures the encrypted message is unique to the encryption key, and does not induce any communication overhead.

Abstract

Modern cryptography, such as Rivest Shamir Adleman (RSA) and Secure Hash Algorithm (SHA), has been designed by humans based on our understanding of cryptographic methods. Neural Network (NN) based cryptography is being investigated due to its ability to learn and implement random cryptographic schemes that may be harder to decipher than human-designed algorithms. NN based cryptography may create a new cryptographic scheme that is NN specific and that changes every time the NN is (re)trained. This is attractive since it would require an adversary to restart its process(es) to learn or break the cryptographic scheme every time the NN is (re)trained. Current challenges facing NN-based encryption include additional communication overhead due to encoding to correct bit errors, quantizing the continuous-valued output of the NN, and enabling One-Time-Pad encryption. With this in mind, the Random Adversarial Data Obfuscation Model (RANDOM) Adversarial Neural Cryptography (ANC) network is introduced. RANDOM is comprised of three new NN layers: the (i) projection layer, (ii) inverse projection layer, and (iii) dot-product layer. This results in an ANC network that (i) is computationally efficient, (ii) ensures the encrypted message is unique to the encryption key, and (iii) does not induce any communication overhead. RANDOM only requires around 100 KB to store and can provide up to 2.5 megabytes per second of end-to-end encrypted communication.

Figures (6)

• Figure 1: Representative illustration of the projection, dot product, and inverse projection layers comprising the RANDOM. The input, $\mathbf{x}$ (blue $\circ$ projected into the output matrix $\mathbf{X}_{W}$(solid, orange $\vartriangle$) by multiplying it with each weight (solid, green $\square$) within the weight matrix $\mathbf{w_{{p}}}$.
• Figure 2: RANDOM's NN architecture for Alice, Bob, and Eve in which projection, inverse projection, and dot-product layers are denoted $P$, $\bar{P}$, and $D$, respectively. A transformer operation is denoted using $T$.
• Figure 3: Average encrypted uniqueness across keys for the RANDOM, CNN google_arvix, and LSTM ANC networks.
• Figure 4: Bit recovery accuracy when using the RANDOM with no quantization, and the CNN google_arvix and LSTM network with a quantization level four, eight, sixteen, and thirty-two.
• Figure 5: Best and worst training accuracy and loss plots when training a RANDOM when all projection and inverse projection layers have a projection dimension of eight.