Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Half-Spaces from Perturbed Contrastive Examples

Aryan Alavi Razavi Ravari, Farnam Mansouri, Yuxin Chen, Valentio Iverson, Adish Singla, Sandra Zilles

TL;DR

The work analyzes learning linear decision boundaries from perturbed contrastive examples, introducing deterministic and probabilistic approximate minimum-distance models (AMDM) that perturb ideal contrastive points via a noise function $f$ tied to the primary point's distance to the boundary. It provides active and passive sample-complexity bounds for 1D thresholds and both homogeneous and general half-spaces under a uniform domain, showing that, for suitable $f$, contrastive feedback speeds learning over traditional membership-query baselines. A key finding is a qualitative separation in the probabilistic model between expected error after a fixed number of queries and the number of samples needed to guarantee a target accuracy, reflecting tail behaviors in the error distribution. Overall, the results demonstrate that perturbed contrastive information can meaningfully accelerate learning in practically relevant settings while retaining non-trivial learning dynamics, with explicit bounds depending on the contraction induced by $f$.

Abstract

We study learning under a two-step contrastive example oracle, as introduced by Mansouri et. al. (2025), where each queried (or sampled) labeled example is paired with an additional contrastive example of opposite label. While Mansouri et al. assume an idealized setting, where the contrastive example is at minimum distance of the originally queried/sampled point, we introduce and analyze a mechanism, parameterized by a non-decreasing noise function $f$, under which this ideal contrastive example is perturbed. The amount of perturbation is controlled by $f(d)$, where $d$ is the distance of the queried/sampled point to the decision boundary. Intuitively, this results in higher-quality contrastive examples for points closer to the decision boundary. We study this model in two settings: (i) when the maximum perturbation magnitude is fixed, and (ii) when it is stochastic. For one-dimensional thresholds and for half-spaces under the uniform distribution on a bounded domain, we characterize active and passive contrastive sample complexity in dependence on the function $f$. We show that, under certain conditions on $f$, the presence of contrastive examples speeds up learning in terms of asymptotic query complexity and asymptotic expected query complexity.

Learning Half-Spaces from Perturbed Contrastive Examples

TL;DR

The work analyzes learning linear decision boundaries from perturbed contrastive examples, introducing deterministic and probabilistic approximate minimum-distance models (AMDM) that perturb ideal contrastive points via a noise function tied to the primary point's distance to the boundary. It provides active and passive sample-complexity bounds for 1D thresholds and both homogeneous and general half-spaces under a uniform domain, showing that, for suitable , contrastive feedback speeds learning over traditional membership-query baselines. A key finding is a qualitative separation in the probabilistic model between expected error after a fixed number of queries and the number of samples needed to guarantee a target accuracy, reflecting tail behaviors in the error distribution. Overall, the results demonstrate that perturbed contrastive information can meaningfully accelerate learning in practically relevant settings while retaining non-trivial learning dynamics, with explicit bounds depending on the contraction induced by .

Abstract

We study learning under a two-step contrastive example oracle, as introduced by Mansouri et. al. (2025), where each queried (or sampled) labeled example is paired with an additional contrastive example of opposite label. While Mansouri et al. assume an idealized setting, where the contrastive example is at minimum distance of the originally queried/sampled point, we introduce and analyze a mechanism, parameterized by a non-decreasing noise function , under which this ideal contrastive example is perturbed. The amount of perturbation is controlled by , where is the distance of the queried/sampled point to the decision boundary. Intuitively, this results in higher-quality contrastive examples for points closer to the decision boundary. We study this model in two settings: (i) when the maximum perturbation magnitude is fixed, and (ii) when it is stochastic. For one-dimensional thresholds and for half-spaces under the uniform distribution on a bounded domain, we characterize active and passive contrastive sample complexity in dependence on the function . We show that, under certain conditions on , the presence of contrastive examples speeds up learning in terms of asymptotic query complexity and asymptotic expected query complexity.
Paper Structure (19 sections, 38 theorems, 145 equations, 1 figure, 2 tables)

This paper contains 19 sections, 38 theorems, 145 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Primary Example Collection.
  4. Contrastive example collection.
  5. Further Notation.
  6. Deterministic Approximate Minimum Distance Model
  7. Active Contrastive Sample Complexity
  8. Passive Contrastive Sample Complexity
  9. Probabilistic Approximate Minimum Distance Model
  10. Active Expected Sample Complexity
  11. Expected Sample Requirement for Accuracy $\varepsilon$
  12. Passive Expected Contrastive Sample Complexity
  13. Conclusion
  14. Active Contrastive Complexity Bounds for Specific Classes
  15. Missing Proofs from Section \ref{['sec:active-noisy']}
  16. ...and 4 more sections

Key Result

Theorem 3.1

Let $f: \mathbb{R}^{\geq 0} \to \mathbb{R}^{\geq 0}$ be a non-decreasing and invertible function Invertibility assumptions are made only for ease of presentation. Without invertibility, all proofs continue to hold by replacing every $h^{-1}(r)$ for any function $h$ with $\inf\{x \ge 0 : h(x) \ge r\} where $\mathbbm I$ denotes the identity function. Then

Figures (1)

  • Figure 1: Geometric interpretation of Lemma \ref{['lem:err-dist']}. Given a primary point $\mathbf x \in \mathcal{X}$, its contrastive example $\mathbf x'$, and a target half-space $C^*$, Panel \ref{['fig:circ_max']} illustrates the configuration maximizing the angle $\angle(\mathbf x',\mathbf x,\mathbf x^{\mathop{\mathrm{proj}}\nolimits})$. Panel \ref{['fig:circ_general']} depicts the hyperplane $C$ induced by the pair $(\mathbf x,\mathbf x')$ in Lemma \ref{['lem:err-dist']}(ii) in the general case, along with the auxiliary half-space $C'$ that is parallel to $C^*$ and passes through $\mathbf x'$. The half-space $C'$ is used to decompose $\mathrm{err}(C,C^*)$; further details are deferred to Appendix \ref{['apdx:active-noisy']}.

Theorems & Definitions (71)

  • Definition 2.1
  • Definition 2.2
  • Theorem 3.1
  • Lemma 3.1
  • Corollary 3.1
  • Theorem 3.2
  • proof
  • Theorem 3.3
  • Theorem 3.4
  • Theorem 3.5
  • ...and 61 more