Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models

TL;DR

The paper models the adoption of a general-purpose AI technology as a two-stage bargaining game between a Generalist $G$ and one or more Domain Specialists $D$, where the initial revenue-share bargain over $\delta$ precedes domain-specific investment that raises performance from $\alpha_0$ to $\alpha_1$. It derives the subgame-perfect equilibria and the Pareto-optimal set of bargains under broad cost/revenue specifications, then provides closed-form and numerical bargaining solutions in the polynomial (notably quadratic) cost regime. The analysis generalizes to $n$ domain specialists, showing that the Pareto set can become complex (potentially disconnected) and yields rich regimes for domain strategies (Contributor, Free-rider, Abstainer) depending on costs and revenues. The findings offer normative frameworks (Vertical Monopoly, Egalitarian, Nash, KS, Max-α1*, etc.) to reason about welfare trade-offs in the deployment of general-purpose models and have implications for incentive design and potential regulation of GPT-like technologies.

Abstract

Recent advances in Machine Learning (ML) and Artificial Intelligence (AI) follow a familiar structure: A firm releases a large, pretrained model. It is designed to be adapted and tweaked by other entities to perform particular, domain-specific functions. The model is described as `general-purpose,' meaning it can be transferred to a wide range of downstream tasks, in a process known as adaptation or fine-tuning. Understanding this process - the strategies, incentives, and interactions involved in the development of AI tools - is crucial for making conclusions about societal implications and regulatory responses, and may provide insights beyond AI about general-purpose technologies. We propose a model of this adaptation process. A Generalist brings the technology to a certain level of performance, and one or more Domain specialist(s) adapt it for use in particular domain(s). Players incur costs when they invest in the technology, so they need to reach a bargaining agreement on how to share the resulting revenue before making their investment decisions. We find that for a broad class of cost and revenue functions, there exists a set of Pareto-optimal profit-sharing arrangements where the players jointly contribute to the technology. Our analysis, which utilizes methods based on bargaining solutions and sub-game perfect equilibria, provides insights into the strategic behaviors of firms in these types of interactions. For example, profit-sharing can arise even when one firm faces significantly higher costs than another. After demonstrating findings in the case of one domain-specialist, we provide closed-form and numerical bargaining solutions in the generalized setting with $n$ domain specialists. We find that any potential domain specialization will either contribute, free-ride, or abstain in their uptake of the technology, and provide conditions yielding these different responses.

Figures (9)

• Figure 1: Illustration of the fine-tuning game. In the first step, players bargain over revenue-sharing agreement $\delta$. In this example, they agree that G will receive $80\%$ of the revenue and D will receive $20\%$. In the second step, $G$ develops the technology to performance level $\alpha_0 = 21$. In the third step, $D$ 'fine-tunes' the technology to $\alpha_1=25$. If the players collectively receive revenue of $25$, they would share so that $G$ receives $20$ and $D$ receives $5$.
• Figure 2: Example to illustrate Theorem \ref{['thm:unimodal-pareto']}. Left: For two strictly unimodal, positive utility functions over a bargaining parameter $\delta$, the set of Pareto-optimal bargaining agreements is the interval between their optima. Within this interval, players' utilities are characterized by a trade-off, whereas outside this interval, any agreement is Pareto-dominated. Right: The non-numerical bargaining outcome ('disagreement') consists in players receiving 0 utility. Thus, if a potential bargaining agreement yields negative utility for one or both players, they prefer to exit the agreement. The relevant set of Pareto-optimal agreements is constrained to only the interval where both players receive payout that exceeds the disagreement scenario. If no such interval exists, disagreement is Pareto-optimal.
• Figure 3: Utilities of the Generalist (left) and Domain Specialist (center) for different bargaining parameters and costs. The general-purpose producer prefers to share revenue when $\frac{c_1}{c_0}<1$ and the domain-specific producer prefers to share revenue when $\frac{c_1}{c_0}>1$. The resulting technology performance $\alpha_1^*$ is depicted on the right. Color bar scales are defined assuming $c_0=1$.
• Figure 4: Various joint-utility functions for finding bargaining solutions. Gray regions are $\delta$ values that are not Pareto-optimal and therefore not candidate bargaining solutions. Color bar scales are defined assuming $c_0=1$.
• Figure 5: Bargaining agreements for the fine-tuning game with quadratic costs. Most bargaining solutions involve revenue sharing, even when one player faces much higher costs.

Theorems & Definitions (92)

• Definition 2.1: Pareto-dominant agreements
• Definition 2.2: Pareto-optimal agreements
• Definition 2.3: Strictly Unimodal Function
• Definition 2.4: Powerful-P solution
• Theorem 2.1
• Theorem 3.1
• Corollary 3.1
• Proposition 3.1
• Proposition 3.2: Powerful-$G$ Solution
• Proposition 3.3: Powerful-$D$ Solution