Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Projected Belief Networks With Discriminative Alignment for Acoustic Event Classification: Rivaling State of the Art CNNs

Paul M. Baggenstoss, Kevin Wilkinghoff, Felix Govaers, Frank Kurth

TL;DR

This work presents Projected Belief Networks (PBNs) as two-directional networks that couple discriminative FFNN classifiers with tractable generative models via a right-inverse sampling path. By applying discriminative alignment (PBN-DA) and optionally integrating an HMM (PBN-DA-HMM), the authors achieve high class selectivity while maintaining generative capabilities, training a separate PBN per class and combining its likelihood with a discriminative objective. On air and underwater acoustic datasets, PBN-DA-HMM matches or exceeds state‑of‑the‑art CNN performance and yields about a 2× error reduction when combined with CNN, illustrating the practical value of integrating generative structure with discriminative learning. The approach offers a cohesive framework for robust acoustic event classification with potential benefits for open-set detection and time-series modeling, supported by reproducible software and data access.

Abstract

The projected belief network (PBN) is a generative stochastic network with tractable likelihood function based on a feed-forward neural network (FFNN). The generative function operates by "backing up" through the FFNN. The PBN is two networks in one, a FFNN that operates in the forward direction, and a generative network that operates in the backward direction. Both networks co-exist based on the same parameter set, have their own cost functions, and can be separately or jointly trained. The PBN therefore has the potential to possess the best qualities of both discriminative and generative classifiers. To realize this potential, a separate PBN is trained on each class, maximizing the generative likelihood function for the given class, while minimizing the discriminative cost for the FFNN against "all other classes". This technique, called discriminative alignment (PBN-DA), aligns the contours of the likelihood function to the decision boundaries and attains vastly improved classification performance, rivaling that of state of the art discriminative networks. The method may be further improved using a hidden Markov model (HMM) as a component of the PBN, called PBN-DA-HMM. This paper provides a comprehensive treatment of PBN, PBN-DA, and PBN-DA-HMM. In addition, the results of two new classification experiments are provided. The first experiment uses air-acoustic events, and the second uses underwater acoustic data consisting of marine mammal calls. In both experiments, PBN-DA-HMM attains comparable or better performance as a state of the art CNN, and attain a factor of two error reduction when combined with the CNN.

Projected Belief Networks With Discriminative Alignment for Acoustic Event Classification: Rivaling State of the Art CNNs

TL;DR

This work presents Projected Belief Networks (PBNs) as two-directional networks that couple discriminative FFNN classifiers with tractable generative models via a right-inverse sampling path. By applying discriminative alignment (PBN-DA) and optionally integrating an HMM (PBN-DA-HMM), the authors achieve high class selectivity while maintaining generative capabilities, training a separate PBN per class and combining its likelihood with a discriminative objective. On air and underwater acoustic datasets, PBN-DA-HMM matches or exceeds state‑of‑the‑art CNN performance and yields about a 2× error reduction when combined with CNN, illustrating the practical value of integrating generative structure with discriminative learning. The approach offers a cohesive framework for robust acoustic event classification with potential benefits for open-set detection and time-series modeling, supported by reproducible software and data access.

Abstract

The projected belief network (PBN) is a generative stochastic network with tractable likelihood function based on a feed-forward neural network (FFNN). The generative function operates by "backing up" through the FFNN. The PBN is two networks in one, a FFNN that operates in the forward direction, and a generative network that operates in the backward direction. Both networks co-exist based on the same parameter set, have their own cost functions, and can be separately or jointly trained. The PBN therefore has the potential to possess the best qualities of both discriminative and generative classifiers. To realize this potential, a separate PBN is trained on each class, maximizing the generative likelihood function for the given class, while minimizing the discriminative cost for the FFNN against "all other classes". This technique, called discriminative alignment (PBN-DA), aligns the contours of the likelihood function to the decision boundaries and attains vastly improved classification performance, rivaling that of state of the art discriminative networks. The method may be further improved using a hidden Markov model (HMM) as a component of the PBN, called PBN-DA-HMM. This paper provides a comprehensive treatment of PBN, PBN-DA, and PBN-DA-HMM. In addition, the results of two new classification experiments are provided. The first experiment uses air-acoustic events, and the second uses underwater acoustic data consisting of marine mammal calls. In both experiments, PBN-DA-HMM attains comparable or better performance as a state of the art CNN, and attain a factor of two error reduction when combined with the CNN.
Paper Structure (38 sections, 1 theorem, 22 equations, 8 figures, 4 tables)

This paper contains 38 sections, 1 theorem, 22 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Generative vs. Discriminative Classification
  3. Attaining the best properties of both approaches
  4. Discriminative and Generative functions in a single network
  5. What's New in This Paper
  6. Review of PDF Projection
  7. Definition of PDF Projection
  8. Data Generation
  9. Chain Rule
  10. Multiple features
  11. Information Maximization by PDF Projection
  12. Projected Belief Network (PBN)
  13. Definition of PBN
  14. Mathematical Details of a PBN Layer
  15. Sampling a PBN
  16. ...and 23 more sections

Key Result

Theorem 1

Let prior $p_{0,x}({\bf x})$ be written as (p0def) , having mean $\lambda({\bf 0})$ as given in (l0def). Then, the surrogate density for $p_m({\bf x}|{\bf z})$ in (jpostdef) is $p_s({\bf x};{\bf W} {\bf h}_z, \alpha_0,\beta)$, where ${\bf h}_z$ is value of ${\bf h}$ that solves Then, as $N$ becomes large, $p_s({\bf x};{\bf W} {\bf h}_z, \alpha_0,\beta) \rightarrow p_m({\bf x}|{\bf z})$. Furthermo

Figures (8)

  • Figure 1: Block diagram of 2-layer PBN in asymptotic form. Several versions of PBN are shown in one figure, as implemented by dotted lines (see text).
  • Figure 2: From top to bottom: PBN trained on class 1 (red), PBN trained on class 2 (blue), PBN trained on class 1 with discriminative alignment, PBN trained on class 2 with discriminative alignment. Left column: input data, center: likelihood surface, right: contour lines of likelihood surface.
  • Figure 3: Mean errors for self-combination as a function of $C$.
  • Figure 4: Mean number of classification errors out of 230 samples, for each of the four partitions when combining PBN-DA-HMM with CNN.
  • Figure 5: Mean number of classification errors out of 230 samples, averaged over the four folds when combining PBN-DA-HMM with CNN.
  • ...and 3 more figures

Theorems & Definitions (1)

  • Theorem 1