Learning through Creation: A Hash-Free Framework for On-the-Fly Category Discovery

Abstract

On-the-Fly Category Discovery (OCD) aims to recognize known classes while simultaneously discovering emerging novel categories during inference, using supervision only from known classes during offline training. Existing approaches rely either on fixed label supervision or on diffusion-based augmentations to enhance the backbone, yet none of them explicitly train the model to perform the discovery task required at test time. It is fundamentally unreasonable to expect a model optimized on limited labeled data to carry out a qualitatively different discovery objective during inference. This mismatch creates a clear optimization misalignment between the offline learning stage and the online discovery stage. In addition, prior methods often depend on hash-based encodings or severe feature compression, which further limits representational capacity. To address these issues, we propose Learning through Creation (LTC), a fully feature-based and hash-free framework that injects novel-category awareness directly into offline learning. At its core is a lightweight, online pseudo-unknown generator driven by kernel-energy minimization and entropy maximization (MKEE). Unlike previous methods that generate synthetic samples once before training, our generator evolves jointly with the model dynamics and synthesizes pseudo-novel instances on the fly at negligible cost. These samples are incorporated through a dual max-margin objective with adaptive thresholding, strengthening the model's ability to delineate and detect unknown regions through explicit creation. Extensive experiments across seven benchmarks show that LTC consistently outperforms prior work, achieving improvements ranging from 1.5 percent to 13.1 percent in all-class accuracy. The code is available at https://github.com/brandinzhang/LTC

Figures (8)

• Figure 1: Comparison of category discovery settings. Unlike NCD/GCD, which rely on a pre-defined query set during training, OCD enables new category discovery directly from streaming data without prior access to test samples.
• Figure 2: Overview of the proposed LTC framework. During training, we extract feature representations from known-class samples and generate pseudo-unknowns via mixup and MKEE perturbation. A dual max-margin loss enforces separation between known and unknown regions, guided by an adaptive threshold. At inference, dynamic prototype matching enables online discovery of novel categories.
• Figure 3: MKEE generates semantically meaningful pseudo-unknowns. (a) Visualization of the feature space on the Oxford Pets dataset, illustrating known-class prototypes and the perturbation trajectories from $x_{\text{mix}}$ to $x_{\text{pus}}$. (b) t-SNE visualization on the CIFAR-10 dataset, where five colored clusters correspond to the five known classes (1–5), and pseudo-unknowns (black triangles) clearly diverge from all known-class regions.
• Figure 4: Hyperparameter sensitivity on the CUB and SCars datasets.$\alpha$ and $\gamma_{mm}$ control the relative contributions of the contrastive and margin-based terms in the total loss $\mathcal{L}_{\text{total}}$.
• Figure 5: Adaptive thresholding improves stability under different initializations. We study the effect of varying the initial similarity threshold $\tau_{\text{init}}$ and the update rate $\beta$ on overall accuracy. Results on CUB and SCars show that adaptive updates ($\beta>0$) maintain stable performance across a wide range of $\tau_{\text{init}}$ values, while fixed thresholds ($\beta=0$) are more sensitive to initialization.