Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring 3D Dataset Pruning

Xiaohan Zhao, Xinyi Shang, Jiacheng Liu, Zhiqiang Shen

TL;DR

This work proposes representation-aware subset selection with per-class retention quotas for long-tail coverage, and prior-invariant teacher supervision using calibrated soft labels and embedding-geometry distillation to control the OA-mAcc trade-off.

Abstract

Dataset pruning has been widely studied for 2D images to remove redundancy and accelerate training, while particular pruning methods for 3D data remain largely unexplored. In this work, we study dataset pruning for 3D data, where its observed common long-tail class distribution nature make optimization under conventional evaluation metrics Overall Accuracy (OA) and Mean Accuracy (mAcc) inherently conflicting, and further make pruning particularly challenging. To address this, we formulate pruning as approximating the full-data expected risk with a weighted subset, which reveals two key errors: coverage error from insufficient representativeness and prior-mismatch bias from inconsistency between subset-induced class weights and target metrics. We propose representation-aware subset selection with per-class retention quotas for long-tail coverage, and prior-invariant teacher supervision using calibrated soft labels and embedding-geometry distillation. The retention quota also serves as a switch to control the OA-mAcc trade-off. Extensive experiments on 3D datasets show that our method can improve both metrics across multiple settings while adapting to different downstream preferences. Our code is available at https://github.com/XiaohanZhao123/3D-Dataset-Pruning.

Exploring 3D Dataset Pruning

TL;DR

This work proposes representation-aware subset selection with per-class retention quotas for long-tail coverage, and prior-invariant teacher supervision using calibrated soft labels and embedding-geometry distillation to control the OA-mAcc trade-off.

Abstract

Dataset pruning has been widely studied for 2D images to remove redundancy and accelerate training, while particular pruning methods for 3D data remain largely unexplored. In this work, we study dataset pruning for 3D data, where its observed common long-tail class distribution nature make optimization under conventional evaluation metrics Overall Accuracy (OA) and Mean Accuracy (mAcc) inherently conflicting, and further make pruning particularly challenging. To address this, we formulate pruning as approximating the full-data expected risk with a weighted subset, which reveals two key errors: coverage error from insufficient representativeness and prior-mismatch bias from inconsistency between subset-induced class weights and target metrics. We propose representation-aware subset selection with per-class retention quotas for long-tail coverage, and prior-invariant teacher supervision using calibrated soft labels and embedding-geometry distillation. The retention quota also serves as a switch to control the OA-mAcc trade-off. Extensive experiments on 3D datasets show that our method can improve both metrics across multiple settings while adapting to different downstream preferences. Our code is available at https://github.com/XiaohanZhao123/3D-Dataset-Pruning.
Paper Structure (28 sections, 6 theorems, 41 equations, 10 figures, 7 tables)

This paper contains 28 sections, 6 theorems, 41 equations, 10 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Theoretical Analysis
  4. Method
  5. Roadmap.
  6. Resolving Term B: Robust Post-pruning Distillation
  7. Optimizing the Structure.
  8. Resolving Term A: Geometry-aware Selection
  9. Experiment
  10. Experiment Setup
  11. Main Results
  12. Additional Results
  13. Conclusion
  14. Proof
  15. Proof for Lemma \ref{['lem:gen_gap']}
  16. ...and 13 more sections

Key Result

Lemma 3.1

Let $\mathcal{G}=\{\ell_{\theta}(\cdot):\theta\in \Theta\}$ and define the discrepancy $D_{\mathcal{G}}(p,q)=\sup_{g\in\mathcal{G}}|\mathbb{E}_{p}[g]-\mathbb{E}_{q}[g]|$. If $\theta^{*}\in\arg\min_{\theta}\mathcal{L}(\theta)$ and $\hat{\theta}\in\arg\min_{\theta}\hat{\mathcal{L}}_{S,w}(\theta)$, the where $D_{\mathcal{G}}$ is an instance of an integral probability metric (IPM) muller1997integral.

Figures (10)

  • Figure 1: Grouped class distribution of ShapeNet55.
  • Figure 2: Illustration of our 3D-Pruner framework, which comprises: (1) base principles (utilizing embedding signals, minimum floor selection, and optimizing structural likelihood in post-pruning training) that remain robust and beneficial across different priors, derived from theoretical analysis of the shared region, and (2) a steering warper that balances between the two priors.
  • Figure 3: Selection composition across algorithms. Incomparable scalar scores (loss, EL2N) bias selection toward many-shot classes. Geometric embedding selection is more robust but still partially affected by class distribution, under-selecting certain classes. SGS mitigates this, also enabling a flexible balancing.
  • Figure 4: Refined Hybrid Selection
  • Figure 5: Comparison of pruning methods across various datasets, models, and budgets on point cloud modality.
  • ...and 5 more figures

Theorems & Definitions (11)

  • Lemma 3.1: Generalization gap via discrepancy
  • Lemma 3.2: Per-class approximation rate (informal)
  • Theorem 3.3: Optimal allocation for term (A)
  • Proposition 4.1: Weight-robustness of KD
  • proof
  • proof
  • Lemma 1.1: Class-wise error bound (formal)
  • proof
  • Theorem 1.2: Optimal allocation for Term (A)
  • proof
  • ...and 1 more