Deep Distributional Learning with Non-crossing Quantile Network

TL;DR

This work introduces a deep distributional learning framework built on non-crossing quantile networks (NQ-Net), enforcing monotonicity of conditional quantiles via a Mean-Gaps architecture with a strictly positive activation on gaps. It establishes non-asymptotic, minimax-optimal convergence guarantees, including robustness to misspecification, Hölder-smooth targets, and a manifold-based reduction of dimensionality, along with distributional RL guarantees under heavy-tailed rewards and dependent data. The theory is complemented by extensive numerical studies showing strong performance relative to existing non-crossing quantile methods and a distributional RL evaluation on Atari games. The results demonstrate that NQ-Net achieves accurate, non-crossing quantile estimates and scalable distributional learning, with practical impact for causal inference, policy evaluation, and robust RL in complex environments.

Abstract

In this paper, we introduce a non-crossing quantile (NQ) network for conditional distribution learning. By leveraging non-negative activation functions, the NQ network ensures that the learned distributions remain monotonic, effectively addressing the issue of quantile crossing. Furthermore, the NQ network-based deep distributional learning framework is highly adaptable, applicable to a wide range of applications, from classical non-parametric quantile regression to more advanced tasks such as causal effect estimation and distributional reinforcement learning (RL). We also develop a comprehensive theoretical foundation for the deep NQ estimator and its application to distributional RL, providing an in-depth analysis that demonstrates its effectiveness across these domains. Our experimental results further highlight the robustness and versatility of the NQ network.

Figures (7)

• Figure 1: A demonstration of quantile crossing on a simulated dataset. The estimated quantile curves at $\tau=0.1, 0.2,\ldots, 0.9$ and the observations are depicted. The left panel presents the estimates from the deep quantile regression without any constraint and there appear crossings. The right figure presents our proposed quantile estimations with non-crossing constraints and there is no crossing.
• Figure 2: A graphical illustration of the Non-Crossing Quantile Network. The "Mean Net" aims to learn the average of all quantiles and the "Gaps Net" aims to learn the differences between adjacent quantiles.
• Figure 3: The simulated univariate models. The sample data with size $N=512$ is depicted as grey dots. Five conditional quantile curves at levels $\tau=$0.05 (blue), 0.25 (orange), 0.5 (green), 0.75 (red), and 0.95 (purple) are depicted as solid curves.
• Figure 4: An instance of the fitted quantile curves under the "Wave" model when $N=512$. The training data is depicted as grey dots. The target (estimated) quantile curves are depicted as dashed (solid) curves at levels $\tau=$0.05 (blue), 0.25 (orange), 0.5 (green), 0.75 (red), 0.95 (purple).
• Figure 5: Comparison of testing scores for NQ-Net* and NC-QR-DQN along the training process.

Theorems & Definitions (10)

• Lemma 1
• Theorem 1
• Definition 1
• Lemma 2: Approximation Error bound
• Lemma 3
• Theorem 2: Non-asymptotic Upper bounds
• Corollary 1
• Lemma 4: Approximation Error bound
• Corollary 2
• Theorem 3