Foreground-Aware Dataset Distillation via Dynamic Patch Selection

TL;DR

The paper tackles the high cost and limited realism of traditional dataset distillation by introducing a foreground-aware pipeline that uses Grounded SAM2 to compute per-image foreground occupancy and category-wise thresholds. It then dynamically decides, on a per-image basis, whether to crop to a single informative patch or resize to preserve foreground-rich regions, followed by a ranking and synthesis step that produces soft-label distilled images. Across CIFAR-10/100, ImageNette, and ImageWoof, with ConvNet and ResNet-18 backbones, the method consistently outperforms baselines, demonstrating improved retention of task-relevant information and robustness to image composition. The approach offers a lightweight, patch-based alternative with strong scalability and applicability to diverse datasets and architectures, paving the way for more efficient distillation without costly optimization loops.

Abstract

In this paper, we propose a foreground-aware dataset distillation method that enhances patch selection in a content-adaptive manner. With the rising computational cost of training large-scale deep models, dataset distillation has emerged as a promising approach for constructing compact synthetic datasets that retain the knowledge of their large original counterparts. However, traditional optimization-based methods often suffer from high computational overhead, memory constraints, and the generation of unrealistic, noise-like images with limited architectural generalization. Recent non-optimization methods alleviate some of these issues by constructing distilled data from real image patches, but the used rigid patch selection strategies can still discard critical information about the main objects. To solve this problem, we first leverage Grounded SAM2 to identify foreground objects and compute per-image foreground occupancy, from which we derive a category-wise patch decision threshold. Guided by these thresholds, we design a dynamic patch selection strategy that, for each image, either selects the most informative patch from multiple candidates or directly resizes the full image when the foreground dominates. This dual-path mechanism preserves more key information about the main objects while reducing redundant background content. Extensive experiments on multiple benchmarks show that the proposed method consistently improves distillation performance over existing approaches, producing more informative and representative distilled datasets and enhancing robustness across different architectures and image compositions.

Figures (6)

• Figure 1: Comparison of generated images obtained by RDED sun2024diversity and our method. The visualization results show that our results retain more critical semantic information of foreground objects.
• Figure 2: Overview of the proposed foreground-aware dataset distillation methodology. In the Foreground Image Recognition stage, Grounded SAM2 is applied to the original dataset $\mathcal{D}$ to obtain foreground masks and per-image foreground occupancy ratios $R_{\text{object}}$, from which category-wise patch decision thresholds $\{\mathcal{T}_i\}$ are computed. In the Dynamic Patch Selection stage, these thresholds guide a dynamic patch selection process that, for each image, either crops multiple candidate patches and selects the one with the highest realism score or directly resizes the full image when the foreground dominates. In the Distillation Data Synthesis Stage, the selected patches are ranked, composed into distilled images via patch concatenation and resizing, and assigned soft labels to form the final distilled dataset $\mathcal{D}_{\text{dist}}$.
• Figure 3: Per-class distributions of foreground object percentage for ImageNette, ImageWoof, and CIFAR-10. The horizontal axis denotes the proportion of foreground pixels in an image (0–100%), and the vertical axis indicates the fraction of images in each class whose foreground occupancy falls within the corresponding bin. The red dashed line in the figure represents the threshold position for this category.
• Figure 4: Visualization results of distilled images for ImageNette.
• Figure 6: Visualization results of distilled images for CIFAR-10.