Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Auditing Parameter-Efficient Fine-Tuning for Few-Shot 3D Medical Image Segmentation

Son Thai Ly, Hien V. Nguyen

TL;DR

SEA-PEFT uses a search-audit-allocate loop that trains active adapters, estimates each adapter's Dice utility by momentarily toggling it off, and then reselects the active set under a parameter budget using a greedy knapsack allocator to automate this process.

Abstract

Adapting foundation models to new clinical sites remains challenging in practice. Domain shift and scarce annotations must be handled by experts, yet many clinical groups do not have ready access to skilled AI engineers to tune adapter designs and training recipes. As a result, adaptation cycles can stretch from weeks to months, particularly in few-shot settings. Existing PEFT methods either require manual adapter configuration or automated searches that are computationally infeasible in few-shot 3D settings. We propose SEA-PEFT (SElf-Auditing Parameter-Efficient Fine-Tuning) to automate this process. SEA-PEFT treats adapter configuration as an online allocation problem solved during fine-tuning rather than through manual, fixed-topology choices. SEA-PEFT uses a search-audit-allocate loop that trains active adapters, estimates each adapter's Dice utility by momentarily toggling it off, and then reselects the active set under a parameter budget using a greedy knapsack allocator. Exponential Moving Average and Interquartile Range smoothing, together with a Finite-State Ranking controller, stabilize the loop and improve reliability in high-noise few-shot regimes. On TotalSegmentator and FLARE'22, SEA-PEFT improves mean Dice by 2.4--2.8 points over the strongest fixed-topology PEFT baselines across 1/5/10-shot settings while training <1% of parameters. For reproducibility purposes, we made our code publicly available at https://github.com/tsly123/SEA_PEFT

Self-Auditing Parameter-Efficient Fine-Tuning for Few-Shot 3D Medical Image Segmentation

TL;DR

SEA-PEFT uses a search-audit-allocate loop that trains active adapters, estimates each adapter's Dice utility by momentarily toggling it off, and then reselects the active set under a parameter budget using a greedy knapsack allocator to automate this process.

Abstract

Adapting foundation models to new clinical sites remains challenging in practice. Domain shift and scarce annotations must be handled by experts, yet many clinical groups do not have ready access to skilled AI engineers to tune adapter designs and training recipes. As a result, adaptation cycles can stretch from weeks to months, particularly in few-shot settings. Existing PEFT methods either require manual adapter configuration or automated searches that are computationally infeasible in few-shot 3D settings. We propose SEA-PEFT (SElf-Auditing Parameter-Efficient Fine-Tuning) to automate this process. SEA-PEFT treats adapter configuration as an online allocation problem solved during fine-tuning rather than through manual, fixed-topology choices. SEA-PEFT uses a search-audit-allocate loop that trains active adapters, estimates each adapter's Dice utility by momentarily toggling it off, and then reselects the active set under a parameter budget using a greedy knapsack allocator. Exponential Moving Average and Interquartile Range smoothing, together with a Finite-State Ranking controller, stabilize the loop and improve reliability in high-noise few-shot regimes. On TotalSegmentator and FLARE'22, SEA-PEFT improves mean Dice by 2.4--2.8 points over the strongest fixed-topology PEFT baselines across 1/5/10-shot settings while training <1% of parameters. For reproducibility purposes, we made our code publicly available at https://github.com/tsly123/SEA_PEFT
Paper Structure (8 sections, 3 theorems, 7 equations, 3 figures, 3 tables)

This paper contains 8 sections, 3 theorems, 7 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Self-Aufit PEFT
  4. Problem Setup and Audit Space
  5. Search--Audit--Allocate Cycle
  6. Experiments
  7. Results and Discussion
  8. Conclusion

Key Result

proposition 1

Let $u_i^{(t)} = \mu_i^{(t)} + \xi_i^{(t)}$ where $\mathbb{E}[\xi_i^{(t)}\!\mid\!\mathcal{F}_{t-1}]=0$, $\mathrm{Var}(\xi_i^{(t)})\leq\sigma^2$, and $\mu_i^{(t)}$ drifts by at most $\delta$ between consecutive audits. After $t$ audits with Exponential Moving Average (EMA) decay $\beta\in(0,1)$: Furthermore, over a sliding history window of audits, the sample Interquartile Range (IQR) $\mathrm{IQR

Figures (3)

  • Figure 1: SEA-PEFT iterates over a PEFT library with three steps: (1) Search selects a small adapter set under a parameter budget; (2) Audit runs lightweight on/off tests to estimate Dice-per-cost utility; (3) Allocate updates the active configuration via budget-aware rank adjustments. This enables rapid adaptation by clinical users without AI engineers.
  • Figure 2: SA-PEFT vs. random PEFT configurations under fixed parameter budgets. Across 1/5/10-shot settings, SEA-PEFT’s gains (Orange) come from principled selection rather than simply using more parameters.
  • Figure 3: Qualitative segmentation comparison between fixed-topology PEFT and SEA-PEFT across eight organs.

Theorems & Definitions (6)

  • proposition 1: Convergence of the EMA+IQR utility estimator
  • proof : Proof sketch
  • lemma 1: Audit coverage
  • proof : Proof sketch
  • proposition 2: FSM reduces structural error
  • proof : Proof sketch