Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Learning Parities with Dependent Noise

Noah Golowich, Ankur Moitra, Dhruv Rohatgi

TL;DR

This note shows that learning parities with small batches of weakly dependent noise, modeled as a $\delta$-Santha–Vazirani source, remains hard under the standard LPN assumption by constructing an explicit reduction. The core idea is to augment the secret-dependent noise with affine terms and then linearize the resulting Bernoulli noise into a linear combination of prior noise terms, enabling a batch LPN sampler (EntangleLPN) that converts standard LPN samples into batch samples with joint noise $p$ in time $\mathrm{poly}(n,2^k)$. This provides a partial converse to the linearization attack and underpins a cryptographic separation between reinforcement learning and supervised learning in prior work, with potential broader applications to learning theory and cryptography. The work raises questions about tightening the dependence on the batch size $k$ and exploring alternative models (e.g., sampling oracles) that could widen the applicability of batch LPN robustness results.

Abstract

In this expository note we show that the learning parities with noise (LPN) assumption is robust to weak dependencies in the noise distribution of small batches of samples. This provides a partial converse to the linearization technique of [AG11]. The material in this note is drawn from a recent work by the authors [GMR24], where the robustness guarantee was a key component in a cryptographic separation between reinforcement learning and supervised learning.

On Learning Parities with Dependent Noise

TL;DR

This note shows that learning parities with small batches of weakly dependent noise, modeled as a -Santha–Vazirani source, remains hard under the standard LPN assumption by constructing an explicit reduction. The core idea is to augment the secret-dependent noise with affine terms and then linearize the resulting Bernoulli noise into a linear combination of prior noise terms, enabling a batch LPN sampler (EntangleLPN) that converts standard LPN samples into batch samples with joint noise in time . This provides a partial converse to the linearization attack and underpins a cryptographic separation between reinforcement learning and supervised learning in prior work, with potential broader applications to learning theory and cryptography. The work raises questions about tightening the dependence on the batch size and exploring alternative models (e.g., sampling oracles) that could widen the applicability of batch LPN robustness results.

Abstract

In this expository note we show that the learning parities with noise (LPN) assumption is robust to weak dependencies in the noise distribution of small batches of samples. This provides a partial converse to the linearization technique of [AG11]. The material in this note is drawn from a recent work by the authors [GMR24], where the robustness guarantee was a key component in a cryptographic separation between reinforcement learning and supervised learning.
Paper Structure (11 sections, 4 theorems, 28 equations)

This paper contains 11 sections, 4 theorems, 28 equations.

Table of Contents

  1. Introduction
  2. When is batch LPN hard?
  3. Related work
  4. Outline.
  5. Proof overview
  6. Solution: adding affine noise (in ${\mathsf{sk}}$).
  7. Linearization.
  8. Proof of \ref{['lemma:construct-corr-lpn']}
  9. Proof of \ref{['lemma:construct-corr-lpn']}.
  10. Discussion
  11. Acknowledgments.

Key Result

Theorem 1.4

There is an algorithm $\mathtt{EntangleLPN}$ with the following property. Fix $n,k \in \mathbb{N}$, $\delta\in(0,1/2^{k+3})$, and $p \in \Delta(\mathbb F_2^k)$. Suppose that $p$ is a $\delta$-Santha-Vazirani source. For every ${\mathsf{sk}} \in \mathbb F_2^n$, for independent samples $(a_i,y_i)_{i=1

Theorems & Definitions (10)

  • Definition 1.1: LPN distribution
  • Definition 1.2: Batch LPN distribution
  • Definition 1.3: c.f. santha1986generating
  • Theorem 1.4: golowich2024exploration
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof