Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Salsa Fresca: Angular Embeddings and Pre-Training for ML Attacks on Learning With Errors

Samuel Stevens, Emily Wenger, Cathy Li, Niklas Nolte, Eshika Saxena, François Charton, Kristin Lauter

TL;DR

This work tackles ML-based attacks on Learning with Errors (LWE) by focusing on sparse binary/ternary secrets at cryptographically relevant dimensions. It introduces Salsa Fresca, combining angular embeddings with an encoder-only transformer and a pre-training regime to drastically reduce preprocessing and data requirements, enabling secret recovery up to $n=1024$ for the first time. The key innovations are faster preprocessing via Flatter with BKZ interleaving, a modular angular embedding that preserves problem structure, and the first application of pre-training to improve sample efficiency in LWE attacks, yielding up to $10\times$ fewer training samples and enabling scalable attacks. These advances significantly push the practical boundaries of ML-based cryptanalysis and have implications for PQC parameter choices, while acknowledging ethical considerations and outlining future improvements across modular arithmetic learning and data generation.

Abstract

Learning with Errors (LWE) is a hard math problem underlying recently standardized post-quantum cryptography (PQC) systems for key exchange and digital signatures. Prior work proposed new machine learning (ML)-based attacks on LWE problems with small, sparse secrets, but these attacks require millions of LWE samples to train on and take days to recover secrets. We propose three key methods -- better preprocessing, angular embeddings and model pre-training -- to improve these attacks, speeding up preprocessing by $25\times$ and improving model sample efficiency by $10\times$. We demonstrate for the first time that pre-training improves and reduces the cost of ML attacks on LWE. Our architecture improvements enable scaling to larger-dimension LWE problems: this work is the first instance of ML attacks recovering sparse binary secrets in dimension $n=1024$, the smallest dimension used in practice for homomorphic encryption applications of LWE where sparse binary secrets are proposed.

Salsa Fresca: Angular Embeddings and Pre-Training for ML Attacks on Learning With Errors

TL;DR

This work tackles ML-based attacks on Learning with Errors (LWE) by focusing on sparse binary/ternary secrets at cryptographically relevant dimensions. It introduces Salsa Fresca, combining angular embeddings with an encoder-only transformer and a pre-training regime to drastically reduce preprocessing and data requirements, enabling secret recovery up to for the first time. The key innovations are faster preprocessing via Flatter with BKZ interleaving, a modular angular embedding that preserves problem structure, and the first application of pre-training to improve sample efficiency in LWE attacks, yielding up to fewer training samples and enabling scalable attacks. These advances significantly push the practical boundaries of ML-based cryptanalysis and have implications for PQC parameter choices, while acknowledging ethical considerations and outlining future improvements across modular arithmetic learning and data generation.

Abstract

Learning with Errors (LWE) is a hard math problem underlying recently standardized post-quantum cryptography (PQC) systems for key exchange and digital signatures. Prior work proposed new machine learning (ML)-based attacks on LWE problems with small, sparse secrets, but these attacks require millions of LWE samples to train on and take days to recover secrets. We propose three key methods -- better preprocessing, angular embeddings and model pre-training -- to improve these attacks, speeding up preprocessing by and improving model sample efficiency by . We demonstrate for the first time that pre-training improves and reduces the cost of ML attacks on LWE. Our architecture improvements enable scaling to larger-dimension LWE problems: this work is the first instance of ML attacks recovering sparse binary secrets in dimension , the smallest dimension used in practice for homomorphic encryption applications of LWE where sparse binary secrets are proposed.
Paper Structure (27 sections, 1 equation, 4 figures, 28 tables)

This paper contains 27 sections, 1 equation, 4 figures, 28 tables.

Table of Contents

  1. Introduction
  2. Context and Attack Overview
  3. Attack Part 1: LWE data preprocessing
  4. Attack Part 2: Model training and secret recovery
  5. Improving upon prior work
  6. Parameters and settings in our work
  7. Data Preprocessing
  8. Model Architecture
  9. Encoder-only model
  10. Angular embedding
  11. Experiments
  12. Training Methods
  13. Training with Fewer Samples
  14. Model Pre-Training
  15. Related Work
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Encoder-only transformer (§\ref{['subsec:encoder-only']}) with angular embedding architecture. See §\ref{['subsec:tokenization']} for an explanation.
  • Figure 2: Count of # successes (orange) and failures (blue) for various NoMod proportions for vocabulary-based and angular embedding schemes.$n=512, \log_2 q =41, h=57\text{-}67$.
  • Figure 3: Mean minimum number of samples needed to recover binary secrets as a function of # pre-training steps. ($n=512$, $\log_2 q=41$, binary secrets).
  • Figure 4: How pre-training affects mean hours to secret recovery for different training dataset sizes. ($n=512$, $\log_2 q=41$, binary secrets). Among secrets recovered by all checkpoints; trend lines calculated using only the pre-trained checkpoints.