Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning from Interval Targets

Rattana Pukdee, Ziqi Ke, Chirag Gupta

TL;DR

This work tackles regression when training targets are given as intervals rather than exact values. It develops two principled approaches: a projection-loss method with non-asymptotic generalization guarantees that leverage smoothness to shrink feasible output ranges, and a min-max framework that optimizes worst-case targets within the intervals, with further refinement when restricting adversaries to the hypothesis class. Theoretical results provide realizable and agnostic generalization bounds that depend on interval quality and the Lipschitz properties of the hypothesis class, while experiments on real-world datasets demonstrate state-of-the-art performance and practical gains from using Lipschitz-constrained models. Overall, the paper connects interval supervision with smoothness-aware learning and robust optimization, offering scalable techniques for learning from imprecise targets in regression tasks.

Abstract

We study the problem of regression with interval targets, where only upper and lower bounds on target values are available in the form of intervals. This problem arises when the exact target label is expensive or impossible to obtain, due to inherent uncertainties. In the absence of exact targets, traditional regression loss functions cannot be used. First, we study the methodology of using a loss functions compatible with interval targets, for which we establish non-asymptotic generalization bounds based on smoothness of the hypothesis class that significantly relaxing prior assumptions of realizability and small ambiguity degree. Second, we propose a novel min-max learning formulation: minimize against the worst-case (maximized) target labels within the provided intervals. The maximization problem in the latter is non-convex, but we show that good performance can be achieved with the incorporation of smoothness constraints. Finally, we perform extensive experiments on real-world datasets and show that our methods achieve state-of-the-art performance.

Learning from Interval Targets

TL;DR

This work tackles regression when training targets are given as intervals rather than exact values. It develops two principled approaches: a projection-loss method with non-asymptotic generalization guarantees that leverage smoothness to shrink feasible output ranges, and a min-max framework that optimizes worst-case targets within the intervals, with further refinement when restricting adversaries to the hypothesis class. Theoretical results provide realizable and agnostic generalization bounds that depend on interval quality and the Lipschitz properties of the hypothesis class, while experiments on real-world datasets demonstrate state-of-the-art performance and practical gains from using Lipschitz-constrained models. Overall, the paper connects interval supervision with smoothness-aware learning and robust optimization, offering scalable techniques for learning from imprecise targets in regression tasks.

Abstract

We study the problem of regression with interval targets, where only upper and lower bounds on target values are available in the form of intervals. This problem arises when the exact target label is expensive or impossible to obtain, due to inherent uncertainties. In the absence of exact targets, traditional regression loss functions cannot be used. First, we study the methodology of using a loss functions compatible with interval targets, for which we establish non-asymptotic generalization bounds based on smoothness of the hypothesis class that significantly relaxing prior assumptions of realizability and small ambiguity degree. Second, we propose a novel min-max learning formulation: minimize against the worst-case (maximized) target labels within the provided intervals. The maximization problem in the latter is non-convex, but we show that good performance can be achieved with the incorporation of smoothness constraints. Finally, we perform extensive experiments on real-world datasets and show that our methods achieve state-of-the-art performance.
Paper Structure (44 sections, 22 theorems, 99 equations, 19 figures, 4 tables)

This paper contains 44 sections, 22 theorems, 99 equations, 19 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Summary of contributions
  3. Related work
  4. Preliminaries and notation
  5. Learning from intervals using a projection loss
  6. Properties of a hypothesis that lie inside the interval targets
  7. Effect of realizability and small ambiguity degree assumptions on $\widetilde{\mathcal{F}}_\eta$
  8. Properties of $\widetilde{\mathcal{F}}_\eta$
  9. Main results
  10. Learning from intervals using a minmax objective
  11. Alternative approaches to solving a minmax objective with constraints
  12. Experiments
  13. Conclusion
  14. Additional related work
  15. Limitations
  16. ...and 29 more sections

Key Result

Proposition 2.1

Suppose that $\ell: \mathcal{Y} \times \mathcal{Y} \to \mathbb{R}$ is a loss function that satisfies Assumption assum: loss then $\pi_\ell(f(x), l , u) = 0$ if and only if $f(x) \in [l,u]$, and we can write

Figures (19)

  • Figure 1: One dimension example of learning from interval targets
  • Figure 2: Smooth hypothesis leads to a smaller interval
  • Figure 4: An interval of $x$ induced by $x'$
  • Figure 5: Upper and lower bound gaps
  • Figure 7: Test MAE of the projection method with Lipschitz MLP using different values of the Lipschitz constant. The vertical line is the Lipschitz constant approximated from the training set. The dashed horizontal lines are the test MAE of PL (Mean) and Projection approach with a standard MLP. Optimal smoothness level leads to a performance gain.
  • ...and 14 more figures

Theorems & Definitions (42)

  • Proposition 2.1
  • Proposition 3.1
  • Definition 3.2: m-Lipschitz
  • Definition 3.3: A lower and upper bound induced by $x'$
  • Proposition 3.4
  • Definition 3.5: A lower and upper bound gap induced by $x'$
  • Theorem 3.6
  • proof
  • Proposition 3.7
  • Theorem 4.1: Generalization bound, Realizable Setting
  • ...and 32 more