Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Throwing Vines at the Wall: Structure Learning via Random Search

Thibault Vatter, Thomas Nagler

TL;DR

The paper tackles vine-copula structure learning, a combinatorially hard problem due to super-exponential candidate structures. It introduces hold-out random search to sample structures and a model confidence-set (MCS) framework to guarantee inclusion of near-optimal candidates, along with an ensemble approach that averages across MCS survivors. Theoretical guarantees accompany the MCS component, and empirical results on diverse real-world datasets show consistent outperformance of existing greedy baselines in density estimation, regression, and probabilistic forecasting, with ensembles providing additional gains. The methods are practical, parallelizable, and implemented in a Python package, offering a scalable path to improved vine-modeling in ML tasks.

Abstract

Vine copulas offer flexible multivariate dependence modeling and have become widely used in machine learning, yet structure learning remains a key challenge. Early heuristics like the greedy algorithm of Dissmann are still considered the gold standard, but often suboptimal. We propose random search algorithms that improve structure selection and a statistical framework based on model confidence sets, which provides theoretical guarantees on selection probabilities and a powerful foundation for ensembling. Empirical results on several real-world data sets show that our methods consistently outperform state-of-the-art approaches.

Throwing Vines at the Wall: Structure Learning via Random Search

TL;DR

The paper tackles vine-copula structure learning, a combinatorially hard problem due to super-exponential candidate structures. It introduces hold-out random search to sample structures and a model confidence-set (MCS) framework to guarantee inclusion of near-optimal candidates, along with an ensemble approach that averages across MCS survivors. Theoretical guarantees accompany the MCS component, and empirical results on diverse real-world datasets show consistent outperformance of existing greedy baselines in density estimation, regression, and probabilistic forecasting, with ensembles providing additional gains. The methods are practical, parallelizable, and implemented in a Python package, offering a scalable path to improved vine-modeling in ML tasks.

Abstract

Vine copulas offer flexible multivariate dependence modeling and have become widely used in machine learning, yet structure learning remains a key challenge. Early heuristics like the greedy algorithm of Dissmann are still considered the gold standard, but often suboptimal. We propose random search algorithms that improve structure selection and a statistical framework based on model confidence sets, which provides theoretical guarantees on selection probabilities and a powerful foundation for ensembling. Empirical results on several real-world data sets show that our methods consistently outperform state-of-the-art approaches.
Paper Structure (47 sections, 3 theorems, 30 equations, 7 figures, 5 tables, 3 algorithms)

This paper contains 47 sections, 3 theorems, 30 equations, 7 figures, 5 tables, 3 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Contributions
  3. Outline
  4. BACKGROUND
  5. Copulas
  6. Vines
  7. Why the Structure Matters
  8. Related Work
  9. General-purpose methods
  10. Regression problems
  11. Sparse models
  12. METHODOLOGY
  13. Setup
  14. Random Search Algorithm
  15. Vine Structure Confidence Sets
  16. ...and 32 more sections

Key Result

Proposition 3.2

Let $M \in \mathds{N}$ be arbitrary and suppose Then alg:MCS with DA-MCS-plug$^2$ yields

Figures (7)

  • Figure 1: Average relative improvement in out-of-sample log-likelihood over Dissmann2013.
  • Figure 2: A vine structure on four variables.
  • Figure 3: Density estimation: Lines indicated how often the Dissmann method is part of the $95\%$-MCS (computed on the training set) across different datasets, estimated over 20 seeds.
  • Figure 4: Relative CPU time of random search methods compared to Dissmann on the Concrete data set.
  • Figure 5: Average computation time of different DA-MCS Python implementations.
  • ...and 2 more figures

Theorems & Definitions (8)

  • Definition 2.1
  • Definition 3.1: MCS for vine structures
  • Proposition 3.2
  • Proposition A.1
  • Lemma B.1
  • proof
  • proof : Proof of \ref{['prop:mcs_marginal']}
  • proof : Proof of Proposition 3.2