Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Iterative Amortized Hierarchical VAE

Simon W. Penninga, Ruud J. G. van Sloun

TL;DR

The paper addresses fast, accurate inference for deep hierarchical VAEs in inverse problems by introducing IA-HVAE, a hybrid framework that combines amortized inference with iterative decoder-gradient refinement. It relies on a linearly separable, transform-domain decoder to enable real-time, top-down latent updates, achieving about a 35x speed-up for deep models while improving reconstruction quality versus vanilla HVAE. The authors demonstrate benefits on CIFAR10 and fastMRI, including tasks such as deblurring and denoising, and highlight the practical impact for real-time inference in complex imaging scenarios. Future work focuses on generalizing the linear decomposition, integrating iterative optimization into training, and extending the approach to sequential measurement problems.

Abstract

In this paper we propose the Iterative Amortized Hierarchical Variational Autoencoder (IA-HVAE), which expands on amortized inference with a hybrid scheme containing an initial amortized guess and iterative refinement with decoder gradients. We achieve this by creating a linearly separable decoder in a transform domain (e.g. Fourier space), enabling real-time applications with very high model depths. The architectural change leads to a 35x speed-up for iterative inference with respect to the traditional HVAE. We show that our hybrid approach outperforms fully amortized and fully iterative equivalents in accuracy and speed respectively. Moreover, the IAHVAE shows improved reconstruction quality over a vanilla HVAE in inverse problems such as deblurring and denoising.

Iterative Amortized Hierarchical VAE

TL;DR

The paper addresses fast, accurate inference for deep hierarchical VAEs in inverse problems by introducing IA-HVAE, a hybrid framework that combines amortized inference with iterative decoder-gradient refinement. It relies on a linearly separable, transform-domain decoder to enable real-time, top-down latent updates, achieving about a 35x speed-up for deep models while improving reconstruction quality versus vanilla HVAE. The authors demonstrate benefits on CIFAR10 and fastMRI, including tasks such as deblurring and denoising, and highlight the practical impact for real-time inference in complex imaging scenarios. Future work focuses on generalizing the linear decomposition, integrating iterative optimization into training, and extending the approach to sequential measurement problems.

Abstract

In this paper we propose the Iterative Amortized Hierarchical Variational Autoencoder (IA-HVAE), which expands on amortized inference with a hybrid scheme containing an initial amortized guess and iterative refinement with decoder gradients. We achieve this by creating a linearly separable decoder in a transform domain (e.g. Fourier space), enabling real-time applications with very high model depths. The architectural change leads to a 35x speed-up for iterative inference with respect to the traditional HVAE. We show that our hybrid approach outperforms fully amortized and fully iterative equivalents in accuracy and speed respectively. Moreover, the IAHVAE shows improved reconstruction quality over a vanilla HVAE in inverse problems such as deblurring and denoising.
Paper Structure (11 sections, 7 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 11 sections, 7 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Iterative Amortized Hierarchical VAE
  4. Implementation details
  5. Iterative optimization
  6. Experiments & Results
  7. Experimental setup
  8. Inference time
  9. Inference quality
  10. Inverse problems
  11. Discussion and Conclusion

Figures (4)

  • Figure 1: Schematic overview of the IA-HVAE architecture. The layers in the orange and purple blocks are shown on the left. A mapping between the model scales and spatial frequency bins is shown on the right for an example with $H\times W = 8\times 8$. Here the DC frequency is generated at the $1\times 1$ scale and the Nyquist frequencies at $8\times 8$. The missing values are filled in with the hermitian conjugate† of the bins that share the same color to produce a symmetric spectrum and a purely real-valued image.
  • Figure 2: Inference time comparison between the vanilla HVAE architecture and the IA-HVAE architecture for different model depths.
  • Figure 3: Visualization of the IA-HVAE (N=25) output at multiple scales for CIFAR10 and fastMRI. Best viewed zoomed in.
  • Figure 4: Qualitative evaluation of the IA-HVAE (N=25) and vanilla HVAE (baseline) for deblurring and denoising experiments.