A Human-Centric Framework for Data Attribution in Large Language Models

TL;DR

The paper addresses how to reframe data attribution for LLMs as a practical, multi-stakeholder problem within a broader data economy. It introduces a human-centric attribution framework that grounds implementation in negotiated outcomes among creators, users, and intermediaries, and connects NLP attribution methods with governance and economic considerations. It surveys existing literature on data governance, attribution techniques, and socioeconomic analyses, and then details a three-step process to translate negotiated goals into concrete attribution units and criteria, including similarity, causality, and usage-based approaches. A moonshot scenario demonstrates how instance-level and training-data attribution could operate in practice, with logs, micro-payments, and transparency features designed to align incentives and reduce plagiarism risk. Overall, the work provides a blueprint for integrating NLP attribution research with governance and market dynamics to cultivate a sustainable, accountable LLM data ecosystem.

Abstract

In the current Large Language Model (LLM) ecosystem, creators have little agency over how their data is used, and LLM users may find themselves unknowingly plagiarizing existing sources. Attribution of LLM-generated text to LLM input data could help with these challenges, but so far we have more questions than answers: what elements of LLM outputs require attribution, what goals should it serve, how should it be implemented? We contribute a human-centric data attribution framework, which situates the attribution problem within the broader data economy. Specific use cases for attribution, such as creative writing assistance or fact-checking, can be specified via a set of parameters (including stakeholder objectives and implementation criteria). These criteria are up for negotiation by the relevant stakeholder groups: creators, LLM users, and their intermediaries (publishers, platforms, AI companies). The outcome of domain-specific negotiations can be implemented and tested for whether the stakeholder goals are achieved. The proposed approach provides a bridge between methodological NLP work on data attribution, governance work on policy interventions, and economic analysis of creator incentives for a sustainable equilibrium in the data economy.

Figures (3)

• Figure 1: The major changes in information flow from the creators to readers/users when LLMs started serving as providers of content, including for assisted production of new materials. In addition to stakeholders discussed in \ref{['sec:stakeholders']}, a relevant group is non-profit organizations who do not serve as intermediaries, but whose outputs (including datasets) may be used by AI industry.
• Figure 2: The human-centric attribution framework is grounded in case-specific stakeholder negotiations, which explicate and balance their expected outcomes and agree on the general criteria for implementing attribution. The example shows one set of possible conflicting objectives for the fact checking use case.
• Figure 3: Moonshot: human-centric data attribution for LLM-assisted creative writing. We show how this process could look like in a user interface for three stakeholders: the LLM user, the creator of the attributed text, and the intermediary party.