Self-Adaptive Cognitive Debiasing for Large Language Models in Decision-Making

TL;DR

The paper tackles the reliability challenge of LLMs in high-stakes decision-making by addressing cognitive biases that emerge in prompts. It introduces Self-Adaptive Cognitive Debiasing (SACD), an iterative three-stage framework (bias determination, bias analysis, cognitive debiasing) that detects, classifies, and removes bias-inducing elements from prompts, enabling effective debiasing in both single-bias and multi-bias settings. Empirical results across finance, healthcare, and legal tasks show SACD achieving the lowest bias scores compared with advanced prompting and prior debiasing methods, with ablations confirming the necessity of bias determination and analysis. The work demonstrates that systematic, iterative bias mitigation substantially improves the trustworthiness and decision quality of LLM-based assistants in real-world domains and suggests avenues for extending debiasing to more biases and applications, as well as exploring pre-training/fine-tuning interventions.

Abstract

Large language models (LLMs) have shown potential in supporting decision-making applications, particularly as personal assistants in the financial, healthcare, and legal domains. While prompt engineering strategies have enhanced the capabilities of LLMs in decision-making, cognitive biases inherent to LLMs present significant challenges. Cognitive biases are systematic patterns of deviation from norms or rationality in decision-making that can lead to the production of inaccurate outputs. Existing cognitive bias mitigation strategies assume that input prompts only contain one type of cognitive bias, limiting their effectiveness in more challenging scenarios involving multiple cognitive biases. To fill this gap, we propose a cognitive debiasing approach, self-adaptive cognitive debiasing (SACD), that enhances the reliability of LLMs by iteratively refining prompts. Our method follows three sequential steps - bias determination, bias analysis, and cognitive debiasing - to iteratively mitigate potential cognitive biases in prompts. We evaluate SACD on finance, healthcare, and legal decision-making tasks using both open-weight and closed-weight LLMs. Compared to advanced prompt engineering methods and existing cognitive debiasing techniques, SACD achieves the lowest average bias scores in both single-bias and multi-bias settings.

Figures (4)

• Figure 1: (a) CoT approach instructs LLMs to “Let’s think step-by-step”, generating intermediate steps between inputs and outputs to improve problem-solving capabilities. However, it overlooks the impact of potential cognitive biases. (b) Self-help methods employ LLMs to rewrite their own prompts directly but fail to perform effectively in multi-bias settings. (c) SACD method iteratively mitigates cognitive biases in prompts by mimicking human debiasing process of bias determination, bias analysis, and cognitive debiasing.
• Figure 2: Iterative performance across finance, healthcare and legal datasets. The backbone LLM is gpt-3.5-turbo.
• Figure 3: Case study for intuitive comparisons in single-bias setting. Green and red represent correct and incorrect results, respectively. Blue denotes cognitive biases in prompts.
• Figure 4: Case study for intuitive comparisons in multi-bias setting. Green and red represent correct and incorrect results, respectively. Blue denotes cognitive biases in prompts.