Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Debiasing LLMs by Fine-tuning

Zhenyu Gao, Wenxi Jiang, Yutong Yan

Abstract

Prior research shows that large language models (LLMs) exhibit systematic extrapolation bias when forming predictions from both experimental and real-world data, and that prompt-based approaches appear limited in alleviating this bias. We propose a supervised fine-tuning (SFT) approach that uses Low-Rank Adaptation (LoRA) to train off-the-shelf LLMs on instruction datasets constructed from rational benchmark forecasts. By intervening at the parameter level, SFT changes how LLMs map observed information into forecasts and thereby mitigates extrapolation bias. We evaluate the fine-tuned model in two settings: controlled forecasting experiments and cross-sectional stock return prediction. In both settings, fine-tuning corrects the extrapolative bias out-of-sample, establishing a low-cost and generalizable method for debiasing LLMs.

Debiasing LLMs by Fine-tuning

Abstract

Prior research shows that large language models (LLMs) exhibit systematic extrapolation bias when forming predictions from both experimental and real-world data, and that prompt-based approaches appear limited in alleviating this bias. We propose a supervised fine-tuning (SFT) approach that uses Low-Rank Adaptation (LoRA) to train off-the-shelf LLMs on instruction datasets constructed from rational benchmark forecasts. By intervening at the parameter level, SFT changes how LLMs map observed information into forecasts and thereby mitigates extrapolation bias. We evaluate the fine-tuned model in two settings: controlled forecasting experiments and cross-sectional stock return prediction. In both settings, fine-tuning corrects the extrapolative bias out-of-sample, establishing a low-cost and generalizable method for debiasing LLMs.

Paper Structure

This paper contains 35 sections, 3 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Use of LLMs in Finance
  3. LLM biases in behavioral economics.
  4. Methodology
  5. Fine-Tuning LLMs
  6. Pre-training.
  7. Alignment.
  8. Prompting.
  9. Our approach.
  10. Debiasing Framework
  11. Bias identification.
  12. Instructional dataset.
  13. Fine-tuning with LoRA.
  14. Out-of-sample evaluation.
  15. Implementation Details
  16. ...and 20 more sections

Figures (6)

  • Figure I: Pre-training and Instructional Tuning Pipeline
  • Figure II: Supervised Fine-Tuning (SFT) Pipeline
  • Figure III: LoRA Reparametrization
  • Figure IV: Debiasing Framework
  • Figure V: AR(1) Exercise: Base Model
  • ...and 1 more figures