Generative Binary Memory: Pseudo-Replay Class-Incremental Learning on Binarized Embeddings

TL;DR

GBM introduces Generative Binary Memory, a pseudo-replay scheme for Class-Incremental Learning that operates on binarized embeddings modeled by Bernoulli Mixture Models. By updating per-class prototypes and generating balanced binary pseudo-exemplars, GBM achieves state-of-the-art incremental accuracy on CIFAR100 and TinyImageNet while reducing memory footprint, and it extends naturally to Binary Neural Networks through embedding binarization. The approach explicitly handles multi-modality in binary feature spaces and supports both Heaviside and Thermometer binarization, enabling efficient deployment on embedded hardware. Overall, GBM provides a principled, memory-efficient alternative to exemplar-replay and latent-replay methods with strong performance gains in both traditional and fully binarized networks.

Abstract

In dynamic environments where new concepts continuously emerge, Deep Neural Networks (DNNs) must adapt by learning new classes while retaining previously acquired ones. This challenge is addressed by Class-Incremental Learning (CIL). This paper introduces Generative Binary Memory (GBM), a novel CIL pseudo-replay approach which generates synthetic binary pseudo-exemplars. Relying on Bernoulli Mixture Models (BMMs), GBM effectively models the multi-modal characteristics of class distributions, in a latent, binary space. With a specifically-designed feature binarizer, our approach applies to any conventional DNN. GBM also natively supports Binary Neural Networks (BNNs) for highly-constrained model sizes in embedded systems. The experimental results demonstrate that GBM achieves higher than state-of-the-art average accuracy on CIFAR100 (+2.9%) and TinyImageNet (+1.5%) for a ResNet-18 equipped with our binarizer. GBM also outperforms emerging CIL methods for BNNs, with +3.1% in final accuracy and x4.7 memory reduction, on CORE50.

Figures (6)

• Figure 1: Motivations of our pseudo-replay CIL approach based on Bernoulli Mixture Model (BMM). a) Illustration of per-class features correlation on binary embedding. b) Illustration of a single prototype encoding and pseudo-examplars generation. Prototype features is computed as the +1's frequency in a class embedding. Pseudo-examplars are generated from the prototype as a Bernoulli process. c) Qualitative investigation of the binary distribution: (left) apparition of modes when reorganising samples and features with hierarchical clustering, (right) t-SNE visualization van2008visualizing of training samples and pseudo-exemplars generated from a BMM.
• Figure 2: Our proposed GBM method for Pseudo Replay on Binary embedding: a) overview of our framework, b) generative procedure from the prototype memory, c) Classifier update on class-balanced batch between new class samples and past class pseudo-exemplars, and d) GBM update with per-class BMM modeling with EM algorithm and prototype concatenation is the memory.
• Figure 3: The 2 proposed approaches for embedding binarization.
• Figure 4: Incremental performance of $GBM^T$ on CIFAR100, $T$=10. a) Comparison against state-of-the-art. b) Training curves on test and training sets. c) Stability-plasticity trade-off, comparing seen class accuracy (Avg) on a subset of New, Past, Initial (Init) classes.
• Figure 5: Embedding visualization on CIFAR100 with $GBM^T_1$. a-b) t-SNE van2008visualizing on 4 classes, samples (light shade) and generated pseudo-exemplars (dark shade). c) Absolute difference between the correlation matrices of exemplars (dataset samples) and pseudo-exemplars.