Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Functional Distribution Networks (FDN)

Omer Haq

TL;DR

An evaluation protocol is proposed that cleanly separates interpolation from extrapolation and stresses OOD sanity checks, and aims to make OOD-aware, well-calibrated neural regression practical and modular.

Abstract

Modern probabilistic regressors often remain overconfident under distribution shift. We present Functional Distribution Networks (FDN), an input-conditioned distribution over network weights that induces predictive mixtures whose dispersion adapts to the input. FDN is trained with a beta-ELBO and Monte Carlo sampling. We further propose an evaluation protocol that cleanly separates interpolation from extrapolation and stresses OOD sanity checks (e.g., that predictive likelihood degrades under shift while in-distribution accuracy and calibration are maintained). On standard regression tasks, we benchmark against strong Bayesian, ensemble, dropout, and hypernetwork baselines under matched parameter and update budgets, and assess accuracy, calibration, and shift-awareness with standard diagnostics. Together, the framework and protocol aim to make OOD-aware, well-calibrated neural regression practical and modular.

Functional Distribution Networks (FDN)

TL;DR

An evaluation protocol is proposed that cleanly separates interpolation from extrapolation and stresses OOD sanity checks, and aims to make OOD-aware, well-calibrated neural regression practical and modular.

Abstract

Modern probabilistic regressors often remain overconfident under distribution shift. We present Functional Distribution Networks (FDN), an input-conditioned distribution over network weights that induces predictive mixtures whose dispersion adapts to the input. FDN is trained with a beta-ELBO and Monte Carlo sampling. We further propose an evaluation protocol that cleanly separates interpolation from extrapolation and stresses OOD sanity checks (e.g., that predictive likelihood degrades under shift while in-distribution accuracy and calibration are maintained). On standard regression tasks, we benchmark against strong Bayesian, ensemble, dropout, and hypernetwork baselines under matched parameter and update budgets, and assess accuracy, calibration, and shift-awareness with standard diagnostics. Together, the framework and protocol aim to make OOD-aware, well-calibrated neural regression practical and modular.
Paper Structure (47 sections, 42 equations, 3 figures, 5 tables, 1 algorithm)

This paper contains 47 sections, 42 equations, 3 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Overview.
  3. Training objective.
  4. Evaluation protocol.
  5. Contributions.
  6. Scope, significance, and modularity.
  7. Position relative to Neural Processes.
  8. Code availability.
  9. Related Work
  10. Positioning.
  11. Method
  12. Preliminaries
  13. Why homoscedastic noise?
  14. FDN: Input-Conditioned Weight Distributions
  15. Variational family (compact).
  16. ...and 32 more sections

Figures (3)

  • Figure 1: Suite of 1D regression evaluations across three functions: (a–c) Risk–Coverage (AURC) curves (lower is better) for the step $\theta(x)$, sine $1.54\,\sin(2.39\,x)$, and quadratic $0.43x^{2}-0.41$; (d–f) corresponding MSE vs. predicted variance scatter plots (dashed line shows the ideal $\mathrm{MSE}=\mathrm{Var}$). Colors denote models; legends within each panel report per-model statistics (e.g., AURC or Spearman’s $\rho$).
  • Figure 2: MSE vs. predicted variance on three 1D regression tasks in a $3\times2$ grid: rows correspond to tasks $\theta(x)$, $1.54\,\sin(2.39\,x)$, and $0.43x^{2}-0.41$; left column shows test interpolation (ID), right column shows test extrapolation (OOD). Colors denote models; legend entries include Spearman’s $\rho$ between variance and MSE. The dashed line marks the ideal relation $\mathrm{MSE}=\mathrm{Var}$.
  • Figure 3: $\Delta MSE$, $\Delta Var$, and $\Delta CRPS$ for three 1D regression tasks: (a) step $\theta(x)$, (b) sine $1.54\,\sin(2.39\,x)$, and (c) quadratic $0.43x^{2}-0.41$.