Dataset Condensation with Color Compensation

TL;DR

DC3 tackles dataset condensation under extreme compression by foregrounding color as both a signal and a semantic unit, addressing color homogenization that plagues pixel-level DD. It combines image-level sample selection with diffusion-based Color Compensation to expand color diversity, aided by clustering-based bin generation and submodular sampling to preserve representativeness. The method achieves state-of-the-art results across ImageNet-1K and several sub-datasets, shows robust cross-architecture generalization, and enables fine-tuning of diffusion-based generators on condensed data with improved generation quality ($\mathrm{FID}$/$\mathrm{IS}$) without model collapse. Overall, DC3 offers a training-free, scalable pathway to high-fidelity, color-rich condensed datasets that support efficient pre-training of large vision models while maintaining semantic integrity; future work will focus on speeding up diffusion-based steps for even faster condensation.

Abstract

Dataset condensation always faces a constitutive trade-off: balancing performance and fidelity under extreme compression. Existing methods struggle with two bottlenecks: image-level selection methods (Coreset Selection, Dataset Quantization) suffer from inefficiency condensation, while pixel-level optimization (Dataset Distillation) introduces semantic distortion due to over-parameterization. With empirical observations, we find that a critical problem in dataset condensation is the oversight of color's dual role as an information carrier and a basic semantic representation unit. We argue that improving the colorfulness of condensed images is beneficial for representation learning. Motivated by this, we propose DC3: a Dataset Condensation framework with Color Compensation. After a calibrated selection strategy, DC3 utilizes the latent diffusion model to enhance the color diversity of an image rather than creating a brand-new one. Extensive experiments demonstrate the superior performance and generalization of DC3 that outperforms SOTA methods across multiple benchmarks. To the best of our knowledge, besides focusing on downstream tasks, DC3 is the first research to fine-tune pre-trained diffusion models with condensed datasets. The Frechet Inception Distance (FID) and Inception Score (IS) results prove that training networks with our high-quality datasets is feasible without model collapse or other degradation issues. Code and generated data are available at https://github.com/528why/Dataset-Condensation-with-Color-Compensation.

Figures (15)

• Figure 1: Dataset Condensation Triangle. Coreset Selection and Dataset Quantization focus on image-level selection, while Dataset Distillation optimizes pixels for better condensation. An ideal DC method should balance information compression and semantic preservation.
• Figure 2: The KDE curves of the normalized RGB pixel value across different DC methods. Previous methods exhibit a Color Homogenization phenomenon in contrast to DC3.
• Figure 3: Pipeline of the DC3 framework. DC3 supports size-scalable, resolution-variable, and architecture-adaptive dataset condensation for downstream tasks and even fine-tuning large vision models (LVMs).
• Figure 4: Comparison between conventional and diffusion-based image processes. Left: Original image. Middle: Visualizations of conventional processing methods: (a) ColorJitter (brightness, contrast, saturation, hue), (b) Grayscale conversions (number of output channels), and (c) Gaussian blur (kernel, $\sigma$). Right: Diffusion-based color compensation. The diffusion-based approach achieves semantic-aware color reasoning, mitigating distortions caused by mathematical transformations in traditional pipelines.
• Figure 5: Visualizations of the relationship between submodular gain and sample distribution. The following situations occur in traditional DQ methods: (a) inconsequential but compact samples may be selected because they are in different bins (bottom-left), and (b) essential and diverse samples may be discarded because they are in the same bins (top-right). Each column represents a different category.