Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Rational Model of Dimension-reduced Human Categorization

Yifan Hong, Chen Wang

TL;DR

This work introduces a dimension-reduced rational framework for human categorization by modeling each category with a prototype and a low-dimensional subspace via a mixture of probabilistic PPCA (mPPCA). It combines a two-level nonparametric prior (CRP/DP) to share principal components across categories, enabling principled few-shot generalization to new categories and subcategories. Theoretical results establish when discarding a PC improves discriminability, and simulations show that dimension reduction can improve or degrade performance depending on information distribution and noise. Empirical validation on CIFAR-10H demonstrates that a single PC per category captures human categorization and correlates with human choices better than full-rank or baseline models, while artificial few-shot experiments reveal context-sensitive generalization consistent with human data. Overall, mPPCA provides a flexible, interpretable account of human categorization that handles high-dimensional stimuli and rapid generalization through dimension-aware representations.

Abstract

Humans can categorize with only a few samples despite the numerous features. To mimic this ability, we propose a novel dimension-reduced category representation using a mixture of probabilistic principal component analyzers (mPPCA). Tests on the ${\tt CIFAR-10H}$ dataset demonstrate that mPPCA with only a single principal component for each category effectively predicts human categorization of natural images. We further impose a hierarchical prior on mPPCA to account for new category generalization. mPPCA captures human behavior in our experiments on images with simple size-color combinations. We also provide sufficient and necessary conditions when reducing dimensions in categorization is rational.

A Rational Model of Dimension-reduced Human Categorization

TL;DR

This work introduces a dimension-reduced rational framework for human categorization by modeling each category with a prototype and a low-dimensional subspace via a mixture of probabilistic PPCA (mPPCA). It combines a two-level nonparametric prior (CRP/DP) to share principal components across categories, enabling principled few-shot generalization to new categories and subcategories. Theoretical results establish when discarding a PC improves discriminability, and simulations show that dimension reduction can improve or degrade performance depending on information distribution and noise. Empirical validation on CIFAR-10H demonstrates that a single PC per category captures human categorization and correlates with human choices better than full-rank or baseline models, while artificial few-shot experiments reveal context-sensitive generalization consistent with human data. Overall, mPPCA provides a flexible, interpretable account of human categorization that handles high-dimensional stimuli and rapid generalization through dimension-aware representations.

Abstract

Humans can categorize with only a few samples despite the numerous features. To mimic this ability, we propose a novel dimension-reduced category representation using a mixture of probabilistic principal component analyzers (mPPCA). Tests on the dataset demonstrate that mPPCA with only a single principal component for each category effectively predicts human categorization of natural images. We further impose a hierarchical prior on mPPCA to account for new category generalization. mPPCA captures human behavior in our experiments on images with simple size-color combinations. We also provide sufficient and necessary conditions when reducing dimensions in categorization is rational.
Paper Structure (43 sections, 2 theorems, 17 equations, 13 figures, 1 table)

This paper contains 43 sections, 2 theorems, 17 equations, 13 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Models of human category learning
  4. Models of human generalization patterns
  5. Dimension-reduced category representation
  6. Theoretical analysis of dimension reduction
  7. Hierarchical prior on feature dimensions
  8. Hierarchical infinite mixture of PPCA
  9. Few-shot generalization
  10. Learning sub-categories
  11. Learning new categories
  12. Simulation studies
  13. Categorization with PPCA representation
  14. Procedure
  15. Results
  16. ...and 28 more sections

Key Result

Proposition 3.1

For given category prototypes $\mu_a,\mu_b$, discarding the $(q+1)$-th PC dimension (for $q=0,1,\dots,q-2$ from the category representation increases the signal-to-noise ratio of $\alpha_q$ ($\text{SNR}_{q}<\text{SNR}_{q+1}$) if and only if

Figures (13)

  • Figure 1: (a) Schematic illustration of the hierarchical prior over PCs. Categories share these components for common variation patterns. (b) Graphical representation of mPPCA.
  • Figure 2: Model accuracy with varying prototype positions, distances and noise structure. When only dimension 2 is informative (the first row), rank-1 PPCA representation is optimal. For equally informative dimensions (the second row), rank-1 representation remains better. But when the noise levels become similar, the performance gap vanishes. (Full results are in \ref{['sim-dimension_reduced_representation']} in the Appendix.)
  • Figure 3: Equal-generalization-probability contour of different models. The two axes correspond to dimensions in the psychological space (e.g., size and color). After learning on axes-aligned clusters, hierarchical models (REFRESH and mPPCA) exhibit knowledge transfer.
  • Figure 4: Model performance with different combinations of feature maps and classifiers. Since these measures are averaged over thousands of images, the error bars are negligible and are not included.
  • Figure 5: Procedure of few-shot generalization experiment. Category 1 or 2 contains semicircles of regular size but varying color. After category learning in session 1, session 2 provides either one-shot or zero-shot instructions. The new category is similar to category 1 and 2 but locate near a different size value. The subcategory is generated from an isotropic Gaussian distribution, aligned with category 2 on the size dimension. Generalization patterns are tested in session 3.
  • ...and 8 more figures

Theorems & Definitions (4)

  • Proposition 3.1
  • Corollary 3.2
  • proof
  • proof