Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis

Xin Zhou, Dingkang Liang, Wei Xu, Xingkui Zhu, Yihan Xu, Zhikang Zou, Xiang Bai

TL;DR

This work tackles the high cost of full fine-tuning in point cloud analysis by proposing Dynamic Adapter with Internal Prompt Tuning (DAPT), a parameter-efficient transfer learning framework. DAPT freezes the backbone and introduces a Dynamic Adapter that generates per-token scales and an Internal Prompt mechanism derived from adapter outputs, enabling instance-aware adaptation with far fewer tunable parameters. Empirical results on benchmarks such as ScanObjectNN and ModelNet40 show that DAPT often matches or surpasses full fine-tuning while reducing trainable parameters by up to 95% and lowering GPU memory usage by around 35%, with competitive or improved accuracy. The approach provides a practical path for adapting large pre-trained 3D models to diverse tasks and datasets while maintaining efficiency and scalability.

Abstract

Point cloud analysis has achieved outstanding performance by transferring point cloud pre-trained models. However, existing methods for model adaptation usually update all model parameters, i.e., full fine-tuning paradigm, which is inefficient as it relies on high computational costs (e.g., training GPU memory) and massive storage space. In this paper, we aim to study parameter-efficient transfer learning for point cloud analysis with an ideal trade-off between task performance and parameter efficiency. To achieve this goal, we freeze the parameters of the default pre-trained models and then propose the Dynamic Adapter, which generates a dynamic scale for each token, considering the token significance to the downstream task. We further seamlessly integrate Dynamic Adapter with Prompt Tuning (DAPT) by constructing Internal Prompts, capturing the instance-specific features for interaction. Extensive experiments conducted on five challenging datasets demonstrate that the proposed DAPT achieves superior performance compared to the full fine-tuning counterparts while significantly reducing the trainable parameters and training GPU memory by 95% and 35%, respectively. Code is available at https://github.com/LMD0311/DAPT.

Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis

TL;DR

This work tackles the high cost of full fine-tuning in point cloud analysis by proposing Dynamic Adapter with Internal Prompt Tuning (DAPT), a parameter-efficient transfer learning framework. DAPT freezes the backbone and introduces a Dynamic Adapter that generates per-token scales and an Internal Prompt mechanism derived from adapter outputs, enabling instance-aware adaptation with far fewer tunable parameters. Empirical results on benchmarks such as ScanObjectNN and ModelNet40 show that DAPT often matches or surpasses full fine-tuning while reducing trainable parameters by up to 95% and lowering GPU memory usage by around 35%, with competitive or improved accuracy. The approach provides a practical path for adapting large pre-trained 3D models to diverse tasks and datasets while maintaining efficiency and scalability.

Abstract

Point cloud analysis has achieved outstanding performance by transferring point cloud pre-trained models. However, existing methods for model adaptation usually update all model parameters, i.e., full fine-tuning paradigm, which is inefficient as it relies on high computational costs (e.g., training GPU memory) and massive storage space. In this paper, we aim to study parameter-efficient transfer learning for point cloud analysis with an ideal trade-off between task performance and parameter efficiency. To achieve this goal, we freeze the parameters of the default pre-trained models and then propose the Dynamic Adapter, which generates a dynamic scale for each token, considering the token significance to the downstream task. We further seamlessly integrate Dynamic Adapter with Prompt Tuning (DAPT) by constructing Internal Prompts, capturing the instance-specific features for interaction. Extensive experiments conducted on five challenging datasets demonstrate that the proposed DAPT achieves superior performance compared to the full fine-tuning counterparts while significantly reducing the trainable parameters and training GPU memory by 95% and 35%, respectively. Code is available at https://github.com/LMD0311/DAPT.
Paper Structure (28 sections, 7 equations, 8 figures, 10 tables, 1 algorithm)

This paper contains 28 sections, 7 equations, 8 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Pre-training on Point Cloud
  4. Parameter-Efficient Transfer Learning
  5. Preliminary
  6. Vision Transformer
  7. Parameter-Efficient Fine-tuning
  8. Methodology
  9. Task-agnostic Feature Transform Strategy
  10. Dynamic Adapter
  11. Internal Prompt Tuning
  12. Analysis
  13. Experiments
  14. Implement Details
  15. 3D Classification
  16. ...and 13 more sections

Figures (8)

  • Figure 1: (a) Adapter tuning utilizes additional residual blocks with manual scale. (b) Prompt tuning usually introduces extra random initialized prompts into the input space. (c) The proposed Dynamic Adapter generates a dynamic scale for each token and seamlessly integrates with Prompt tuning. (d) The proposed method achieves an ideal trade-off between tunable parameters and performance.
  • Figure 2: (a) The pipeline of our DAPT. During training, we fix the entire backbone, only fine-tuning the newly added parameters. (b) The detail of the Dynamic Adapter with Internal Prompt. (c) Comparison of GPU memory usage among full fine-tuning, IDPT zha2023instance and ours.
  • Figure 3: Effect of different inputs for downstream task head. We conduct experiments on the hardest variant (i.e., PB_T50_RS) of ScanObjectNN uy2019revisiting with Point-MAE pang2022masked baseline.
  • Figure 4: A visual results of the average value of $S_d$ and the ratio of adjusted tokens for each layer across three samples in OBJ_ONLY variant of ScanObjectNN uy2019revisiting.
  • Figure 5: The effect of the number of Internal Prompts.
  • ...and 3 more figures