Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models

Yongchan Kwon, Eric Wu, Kevin Wu, James Zou

TL;DR

The theoretical analysis shows that DataInf is particularly well-suited for parameter-efficient fine-tuning techniques such as LoRA, and systematic empirical evaluations show that it accurately approximates influence scores and is orders of magnitude faster than existing methods.

Abstract

Quantifying the impact of training data points is crucial for understanding the outputs of machine learning models and for improving the transparency of the AI pipeline. The influence function is a principled and popular data attribution method, but its computational cost often makes it challenging to use. This issue becomes more pronounced in the setting of large language models and text-to-image models. In this work, we propose DataInf, an efficient influence approximation method that is practical for large-scale generative AI models. Leveraging an easy-to-compute closed-form expression, DataInf outperforms existing influence computation algorithms in terms of computational and memory efficiency. Our theoretical analysis shows that DataInf is particularly well-suited for parameter-efficient fine-tuning techniques such as LoRA. Through systematic empirical evaluations, we show that DataInf accurately approximates influence scores and is orders of magnitude faster than existing methods. In applications to RoBERTa-large, Llama-2-13B-chat, and stable-diffusion-v1.5 models, DataInf effectively identifies the most influential fine-tuning examples better than other approximate influence scores. Moreover, it can help to identify which data points are mislabeled.

DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models

TL;DR

The theoretical analysis shows that DataInf is particularly well-suited for parameter-efficient fine-tuning techniques such as LoRA, and systematic empirical evaluations show that it accurately approximates influence scores and is orders of magnitude faster than existing methods.

Abstract

Quantifying the impact of training data points is crucial for understanding the outputs of machine learning models and for improving the transparency of the AI pipeline. The influence function is a principled and popular data attribution method, but its computational cost often makes it challenging to use. This issue becomes more pronounced in the setting of large language models and text-to-image models. In this work, we propose DataInf, an efficient influence approximation method that is practical for large-scale generative AI models. Leveraging an easy-to-compute closed-form expression, DataInf outperforms existing influence computation algorithms in terms of computational and memory efficiency. Our theoretical analysis shows that DataInf is particularly well-suited for parameter-efficient fine-tuning techniques such as LoRA. Through systematic empirical evaluations, we show that DataInf accurately approximates influence scores and is orders of magnitude faster than existing methods. In applications to RoBERTa-large, Llama-2-13B-chat, and stable-diffusion-v1.5 models, DataInf effectively identifies the most influential fine-tuning examples better than other approximate influence scores. Moreover, it can help to identify which data points are mislabeled.
Paper Structure (45 sections, 1 theorem, 20 equations, 7 figures, 7 tables, 1 algorithm)

This paper contains 45 sections, 1 theorem, 20 equations, 7 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Preliminaries
  4. Influence Function
  5. Influence function for deep neural network models
  6. LiSSA
  7. DataInf: Efficient Influence Computation
  8. Approximation error analysis
  9. Experiments
  10. Experimental settings
  11. Approximation error analysis
  12. Results
  13. Mislabeled data detection
  14. Results
  15. Influential data identification
  16. ...and 30 more sections

Key Result

Theorem 1

Suppose $\max_{i\in[n]} \norm{\nabla_{\theta_l} \ell_i}_{\infty}$ and $\lambda_l$ are bounded. Then, the spectral norm of the difference $\norm{ \left( \frac{1}{n} \sum_{i=1}^n S_{li} \right)^{-1} - \frac{1}{n} \sum_{i=1}^n S_{li}^{-1} }_2$ is bounded by $O( d_l ^2)$.

Figures (7)

  • Figure 1: Correlation coefficient comparison of the three influence computation methods. The correlation coefficient captures the similarity to the exact computation of the influence function (equation \ref{['eqn:influence_function_for_dnn']}), and thus the higher the correlation coefficient, the better. The error bar indicates a 95% confidence interval based on 20 independent runs. DataInf is significantly more correlated with the exact influence values than other methods for all ranks $r \in \{1,2,4\}$, showing better approximation ability. Also, the correlation coefficient of DataInf generally decreases as the rank increases, consistent with our theoretical analysis.
  • Figure 2: Mislabeled data detection ability comparison of the four influence computation methods when the rank of LoRA matrix $r$ is $4$. The detection ability is evaluated with AUC, and the error bar indicates a 95% confidence interval based on 20 independent runs. DataInf shows better than or comparable detection ability to Exact, and it significantly outperforms Hessian-free and LiSSA on all four datasets. As for the runtime, DataInf is much faster than Exact, demonstrating the practical effectiveness of our method.
  • Figure 3: Illustrative examples of most and least influential training data points discovered using DataInf across the text generation and text-to-image generation tasks performed with the Llama-2-13B-chat and stable-diffusion-v1.5 models. The most (resp. least) influential data point has the largest (resp. smallest) absolute influence on the test example among training data points. DataInf has successfully identified the most influential data points, which exhibit a high degree of relevance to test example prompts. Conversely, the least influential data points identified by DataInf demonstrate lower relevance. In essence, DataInf is effective at detecting influential data points.
  • Figure 4: Examples of images used in the text-to-image generation task, along with before and after images from the LoRA fine-tuning of the stable-diffusion-v1.5 model.
  • Figure 5: Correlation coefficient comparison of the three influence computation methods when data are clean. The experimental settings are exactly the same as the one in Figure \ref{['fig:approximation_error_analysis']} except for the presence of noisy data.
  • ...and 2 more figures

Theorems & Definitions (2)

  • Theorem 1: Approximation error analysis
  • proof