Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conformalized High-Density Quantile Regression via Dynamic Prototypes-based Probability Density Estimation

Batuhan Cengiz, Halil Faruk Karagoz, Tufan Kumbasar

TL;DR

This work introduces a conformalized high-density quantile regression approach with a dynamically adaptive set of prototypes that consistently achieves high-quality prediction regions with enhanced coverage and robustness, all while utilizing fewer prototypes and memory, ensuring scalability to higher dimensions.

Abstract

Recent methods in quantile regression have adopted a classification perspective to handle challenges posed by heteroscedastic, multimodal, or skewed data by quantizing outputs into fixed bins. Although these regression-as-classification frameworks can capture high-density prediction regions and bypass convex quantile constraints, they are restricted by quantization errors and the curse of dimensionality due to a constant number of bins per dimension. To address these limitations, we introduce a conformalized high-density quantile regression approach with a dynamically adaptive set of prototypes. Our method optimizes the set of prototypes by adaptively adding, deleting, and relocating quantization bins throughout the training process. Moreover, our conformal scheme provides valid coverage guarantees, focusing on regions with the highest probability density. Experiments across diverse datasets and dimensionalities confirm that our method consistently achieves high-quality prediction regions with enhanced coverage and robustness, all while utilizing fewer prototypes and memory, ensuring scalability to higher dimensions. The code is available at https://github.com/batuceng/max_quantile .

Conformalized High-Density Quantile Regression via Dynamic Prototypes-based Probability Density Estimation

TL;DR

This work introduces a conformalized high-density quantile regression approach with a dynamically adaptive set of prototypes that consistently achieves high-quality prediction regions with enhanced coverage and robustness, all while utilizing fewer prototypes and memory, ensuring scalability to higher dimensions.

Abstract

Recent methods in quantile regression have adopted a classification perspective to handle challenges posed by heteroscedastic, multimodal, or skewed data by quantizing outputs into fixed bins. Although these regression-as-classification frameworks can capture high-density prediction regions and bypass convex quantile constraints, they are restricted by quantization errors and the curse of dimensionality due to a constant number of bins per dimension. To address these limitations, we introduce a conformalized high-density quantile regression approach with a dynamically adaptive set of prototypes. Our method optimizes the set of prototypes by adaptively adding, deleting, and relocating quantization bins throughout the training process. Moreover, our conformal scheme provides valid coverage guarantees, focusing on regions with the highest probability density. Experiments across diverse datasets and dimensionalities confirm that our method consistently achieves high-quality prediction regions with enhanced coverage and robustness, all while utilizing fewer prototypes and memory, ensuring scalability to higher dimensions. The code is available at https://github.com/batuceng/max_quantile .

Paper Structure

This paper contains 13 sections, 25 equations, 7 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. CHDQR Framework
  3. Regression-as-Classification for density estimation
  4. Developing the Loss function
  5. Dynamic Number of Prototypes
  6. Conformalized High Density Quantile Regression
  7. Experiments
  8. Metrics
  9. 1D Dataset Results
  10. 2d Dataset Results
  11. Robustness to outliers
  12. Discussion & Conclusion
  13. Sythetic Data Details

Figures (7)

  • Figure 1: A simple demonstration for advantages of our method CHDQR. (a) Quantile Regression generates wide bands. (b) Regression-as-classification approach uses static bins, disregarding data distribution. (c) CHDQR dynamically updates prototypes toward data and discretizes dense regions with higher precision.
  • Figure 2: Êffect of different Loss functions. (a) Cross-entropy loss closes the distance between target distribution $q$ and the prediction $\hat{P}$. (b) Shows how $\mathbb{L}_{q}$ moves closest prototype toward $y$. (c) Shows $\delta$ threshold of $\mathbb{L}_{rep}$ and how it pushes prototypes from each other within range.
  • Figure 3: Visualization of Uncond1d Dataset for 90% Coverage
  • Figure 4: Visualization of Unconditional1d Dataset for 50% Coverage
  • Figure 5: Visualization of Unconditional1d Dataset for 10% Coverage
  • ...and 2 more figures