Quantification via Gaussian Latent Space Representations

TL;DR

This work tackles prevalence estimation under prior probability shift by introducing GMNet, an end-to-end neural network that represents bags via a Bag Representation Module grounded in a Gaussian latent space. The bag representation is formed by mean likelihoods across K Gaussians across multiple latent spaces, enabling a rich, permutation-invariant encoding that directly supports a quantification loss. The model employs parameter initialization tailored to Gaussian mixtures, Latent Space Similarity Regularization via Centered Kernel Alignment, and Bag Mixer APP data augmentation to improve generalization, achieving state-of-the-art results on LeQua multiclass datasets T1B/T2 and strong performance on the ordinal T3. These results demonstrate the practical advantage of end-to-end bag-based quantification, data-efficient training, and potential applicability to Learning from Label Proportions and other set-processing tasks.

Abstract

Quantification, or prevalence estimation, is the task of predicting the prevalence of each class within an unknown bag of examples. Most existing quantification methods in the literature rely on prior probability shift assumptions to create a quantification model that uses the predictions of an underlying classifier to make optimal prevalence estimates. In this work, we present an end-to-end neural network that uses Gaussian distributions in latent spaces to obtain invariant representations of bags of examples. This approach addresses the quantification problem using deep learning, enabling the optimization of specific loss functions relevant to the problem and avoiding the need for an intermediate classifier, tackling the quantification problem as a direct optimization problem. Our method achieves state-of-the-art results, both against traditional quantification methods and other deep learning approaches for quantification. The code needed to reproduce all our experiments is publicly available at https://github.com/AICGijon/gmnet.

Figures (5)

• Figure 1: Basic network architecture for a problem with three classes. The network processes a bag of examples and projects it into a latent space through a Feature Extraction Module (FEM). Then, an order invariant representation of the bag is computed by the Bag Representation Module (BRM). In the figure, Gaussian distributions are used for obtaining the bag representation (as explained in Section \ref{['sec:latentrepr']}). Other permutation invariant layers as basic pooling layers or differentiable histograms might be used instead to represent the bags. Finally, the Quantification Module (QM), composed by a set of fully connected layers, relates the bag representation vector to the actual class prevalences. Note that the FEM can be adapted to the problem at hand (CNN for images, Transformers for text, etc).
• Figure 2: Extension for using multiple latent spaces representations. Each latent space $\mathbf{Z}_l$ can have a different number of dimensions $d$, and a different number $K$ of Gaussian distributions to model it (hyperparameters). After the examples in the bag are projected to each latent space, we compute the likelihood $p^{(l)}(\mathbf{z}_i|k)$ of each example $i$ to belong to each Gaussian distribution $k=1..K$ in a latent space $l=1..L$. Then we apply normalization and the mean by Gaussian, getting a vector $\mathbf{r}_l$. The final representation layer is the concatenation of each vector $\mathbf{r}_l$.
• Figure 3: Example of the BRM inner workings using a bare-bones architecture. Input data corresponds to a synthetic multiclass problem with four classes and two input dimensions. Data is projected to a latent space just using a sigmoid function (note that in the full network, the FEM will be in charge of projecting input data to the latent space). For the easiness of visualization, just three Gaussian distributions are in charge of modeling input data. As training progresses, the network learns to place the Gaussian distributions in convenient places to create a useful representation $\mathbf{r}$ of the bag, that the QM can use with the objective of minimizing the chosen loss function (L1 in this case). Note that the number of classes and the number of Gaussian distributions are not directly related, as the network will learn their centers and shapes to optimally represent the data (in the experiments, we will use 100 Gaussians for problems with up to 28 classes).
• Figure 4: U+APP setting over the T2 dataset.
• Figure 5: U setting over the T2 dataset.